Nov 04 00:54:04.991235 ip-10-0-136-128 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 04 00:54:04.991243 ip-10-0-136-128 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 04 00:54:04.991250 ip-10-0-136-128 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 04 00:54:04.991468 ip-10-0-136-128 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 04 00:54:15.146371 ip-10-0-136-128 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 04 00:54:15.146391 ip-10-0-136-128 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c80544fb1225444ead0422e8282c30b3 --
Nov 04 00:56:28.941035 ip-10-0-136-128 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 00:56:29.341881 ip-10-0-136-128 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 00:56:29.341881 ip-10-0-136-128 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 00:56:29.341881 ip-10-0-136-128 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 00:56:29.341881 ip-10-0-136-128 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 00:56:29.344858 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344781    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.344858 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344858    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344862    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344866    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344870    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344873    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344875    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344878    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344881    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344884    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344887    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344889    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344892    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344895    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344898    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344901    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344904    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344906    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344909    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344912    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344914    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.344929 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344917    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344919    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344922    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344925    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344927    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344930    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344933    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344935    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344938    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344941    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344944    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344946    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344949    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344952    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344955    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344957    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344960    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344962    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344965    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.345403 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344969    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344972    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344974    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344977    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344979    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344982    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344985    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344988    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344990    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344993    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344995    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.344998    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345000    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345003    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345007    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345011    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345016    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345019    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345022    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.345870 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345024    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345027    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345030    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345032    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345035    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345038    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345040    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345043    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345045    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345048    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345050    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345053    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345055    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345058    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345061    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345063    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345066    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345068    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345071    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345073    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.346323 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345077    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345079    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345082    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345084    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345087    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345089    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345092    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345096    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345099    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345101    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345104    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345106    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345112    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345116    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345119    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345122    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345125    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345128    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345130    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.346812 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345133    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345135    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345138    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345141    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345144    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345155    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345158    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.345160    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346839    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346848    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346851    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346854    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346857    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346859    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346866    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346870    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346873    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346875    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346878    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346880    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.347255 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346883    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346885    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346888    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346890    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346893    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346897    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346902    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346906    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346909    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346912    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346915    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346918    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346921    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346923    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346926    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346929    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346932    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346935    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346937    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.347757 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346940    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346942    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346945    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346948    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346950    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346953    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346955    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346958    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346961    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346963    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346966    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346968    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346971    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346973    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346976    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346978    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346980    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346983    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346986    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346989    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.348209 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346992    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346994    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.346997    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347000    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347002    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347004    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347007    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347010    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347012    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347015    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347018    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347021    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347023    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347026    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347028    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347031    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347033    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347036    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347038    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347041    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.348686 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347044    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347047    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347049    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347052    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347055    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347058    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347060    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347063    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347066    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347069    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347071    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347075    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347079    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347082    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347085    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347087    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347090    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347092    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347095    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.349191 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347098    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347101    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347103    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347105    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347108    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347110    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347113    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347115    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347118    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347120    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347123    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347125    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347128    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347130    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347133    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347135    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347623    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347628    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347631    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.349647 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347634    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347637    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347640    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347643    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347645    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347648    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347651    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347653    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347656    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347659    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347661    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347664    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347669    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347672    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347675    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347677    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347680    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347683    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347685    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.350152 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347688    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347691    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347693    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347696    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347698    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347701    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347704    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347706    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347709    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347712    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347714    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347717    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347720    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347722    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347725    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347747    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347751    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347754    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347756    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347759    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.350607 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347762    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347765    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347767    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347771    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347773    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347779    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347782    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347786    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347788    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347791    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347793    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347796    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347799    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347801    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347804    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347806    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347809    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347812    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347814    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.351195 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347817    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347819    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347822    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347825    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347828    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347830    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347833    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347835    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347838    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347840    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347843    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347847    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347850    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347853    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347855    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347858    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347860    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347862    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347866    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.351648 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347868    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347871    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347873    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347876    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347878    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347881    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347883    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347886    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347888    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347893    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347895    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347898    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347900    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347902    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347905    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347908    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347911    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347914    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347917    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347919    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.352120 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347922    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347924    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347926    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347929    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347931    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.347934    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348835    2581 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348854    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348866    2581 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348872    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348877    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348880    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348884    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348889    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348892    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348896    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348899    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348903    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348906    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348909    2581 flags.go:64] FLAG: --cgroup-root=""
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348911    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348917    2581 flags.go:64] FLAG: --client-ca-file=""
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348920    2581 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 00:56:29.352627 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348923    2581 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348927    2581 flags.go:64] FLAG: --cluster-domain=""
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348929    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348933    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348936    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348939    2581 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348943    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348947    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348951    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348954    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348957    2581 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348960    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348963    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348966    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348969    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348973    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348976    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348979    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348982    2581 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348986    2581 flags.go:64] FLAG: --enable-server="true"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348989    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348994    2581 flags.go:64] FLAG: --event-burst="100"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.348998    2581 flags.go:64] FLAG: --event-qps="50"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349001    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 00:56:29.353220 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349004    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349007    2581 flags.go:64] FLAG: --eviction-hard=""
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349012    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349015    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349018    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349021    2581 flags.go:64] FLAG: --eviction-soft=""
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349025    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349028    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349031    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349034    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349037    2581 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349040    2581 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349042    2581 flags.go:64] FLAG: --feature-gates=""
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349046    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349050    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349053    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349057    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349060    2581 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349063    2581 flags.go:64] FLAG: --help="false"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349066    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349069    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349072    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349075    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 00:56:29.353867 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349079    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349082    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349085    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349088    2581 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349092    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349095    2581 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349098    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349101    2581 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349104    2581 flags.go:64] FLAG: --kube-reserved=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349107    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349110    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349113    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349116    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349119    2581 flags.go:64] FLAG: --lock-file=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349121    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349125    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349129    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349134    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349137    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349140    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349143    2581 flags.go:64] FLAG: --logging-format="text"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349154    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349158    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349161    2581 flags.go:64] FLAG: --manifest-url=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349164    2581 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 00:56:29.354433 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349169    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349173    2581 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349177    2581 flags.go:64] FLAG: --max-pods="110"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349180    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349183    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349186    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349188    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349192    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349195    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349198    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349206    2581 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349210    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349213    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349216    2581 flags.go:64] FLAG: --pod-cidr=""
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349219    2581 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349222    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349225    2581 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349228    2581 flags.go:64] FLAG: --port="10250"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349231    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349234    2581 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0e6c45a8a47a38687"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349237    2581 flags.go:64] FLAG: --qos-reserved=""
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349240    2581 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349244    2581 flags.go:64] FLAG: --register-node="true"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349247    2581 flags.go:64] FLAG: --register-with-taints=""
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349251    2581 flags.go:64] FLAG: --registry-burst="10"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349254    2581 flags.go:64] FLAG: --registry-qps="5"
Nov 04 00:56:29.355038 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349256    2581 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349259    2581 flags.go:64] FLAG: --reserved-memory=""
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349263    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349266    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349269    2581 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349272    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349275    2581 flags.go:64] FLAG: --runonce="false"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349278    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349281    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349284    2581 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349287    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349290    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349293    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349296    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349299    2581 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349302    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349305    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349309    2581 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349312    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349315    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349318    2581 flags.go:64] FLAG: --system-cgroups=""
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349321    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349326    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349329    2581 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349332    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 00:56:29.355651 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349336    2581 flags.go:64] FLAG: --tls-min-version=""
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349339    2581 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349342    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349345    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349348    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349351    2581 flags.go:64] FLAG: --v="2"
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349357    2581 flags.go:64] FLAG: --version="false"
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349361    2581 flags.go:64] FLAG: --vmodule=""
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349365    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349368    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349509    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349512    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349515    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349518    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349520    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349523    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349526    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349529    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349531    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349534    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349536    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.356309 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349539    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349541    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349544    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349548    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349551    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349553    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349556    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349558    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349561    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349563    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349566    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349569    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349572    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349575    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349577    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349580    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349582    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349585    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349588    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349590    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.356828 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349592    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349595    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349598    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349600    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349603    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349605    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349608    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349610    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349613    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349616    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349618    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349621    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349623    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349626    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349628    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349632    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349635    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349637    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349640    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349642    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349644    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.357307 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349647    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349650    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349652    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349655    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349658    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349661    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349663    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349666    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349668    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349671    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349673    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349676    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349678    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349681    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349683    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349686    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349688    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349691    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349693    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349695    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.357869 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349698    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349701    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349704    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349706    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349709    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349711    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349716    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349719    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349722    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349724    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349742    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349746    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349749    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349752    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349756    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349761    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349764    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349768    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.358342 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349771    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349774    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349777    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349779    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349782    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349785    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349787    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349790    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349792    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349795    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349797    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349800    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349802    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349805    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349807    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.358815 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349810    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.349837    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349872    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349878    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349881    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349885    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349890    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349893    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349895    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349898    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349901    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349903    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349906    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349908    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349911    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349914    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.359207 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349917    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349919    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349922    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349925    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349927    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349930    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349932    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349935    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349937    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349940    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349943    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349945    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349948    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349950    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349953    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349955    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349958    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349962    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349964    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349967    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349969    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.359590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349975    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349979    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349982    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349984    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349988    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349990    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349993    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349995    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.349998    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350001    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350003    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350006    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350008    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350011    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350013    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350016    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350018    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350021    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350024    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350026    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.360098 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350029    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350031    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350034    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350036    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350039    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350041    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350043    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350046    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350048    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350052    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350054    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350057    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350061    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350065    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350068    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350071    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350073    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350076    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350080    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.360575 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350084    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350087    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350091    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350094    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350097    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350100    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350103    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350105    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350108    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350111    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350114    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350117    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350119    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350122    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350125    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350127    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350130    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350132    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.361052 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350135    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350137    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350140    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350142    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350145    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350148    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350150    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350154    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350158    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350161    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350164    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350166    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350169    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.350171    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.357798    2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.357813    2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.358386    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.358392    2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358443    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.361551 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358448    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358452    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358455    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358458    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358461    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358464    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358467    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358470    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358473    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358476    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358478    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358481    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358484    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358486    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358489    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358492    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358494    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358497    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358499    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358502    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.362213 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358505    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358507    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358510    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358512    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358515    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358518    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358520    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358523    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358527    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358529    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358532    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358535    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358537    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358540    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358542    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358545    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358547    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358550    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358552    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358555    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.362709 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358558    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358560    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358562    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358565    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358567    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358570    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358572    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358575    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358577    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358580    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358582    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358585    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358588    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358590    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358593    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358596    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358598    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358601    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358603    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358606    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.363236 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358609    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358612    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358614    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358617    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358619    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358622    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358624    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358627    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358630    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358632    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358635    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358637    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358640    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358642    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358645    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358647    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358650    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358652    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358655    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358658    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.363710 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358660    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358663    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358665    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358668    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358670    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358675    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358678    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358681    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358684    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358686    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358689    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358691    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358694    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358696    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358699    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358702    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358704    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358708    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358711    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.364273 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358714    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358716    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358719    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358722    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358724    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358727    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.358766    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358888    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358893    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358896    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358899    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358902    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358904    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358907    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358910    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358912    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.364715 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358915    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358917    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358920    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358923    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358925    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358928    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358930    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358933    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358936    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358939    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358941    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358944    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358947    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358950    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358952    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358955    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358957    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358960    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358962    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358965    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.365130 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358967    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358970    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358972    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358975    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358978    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358980    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358983    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358985    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358988    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358990    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358992    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358995    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.358998    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359000    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359002    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359005    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359007    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359010    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359012    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359015    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.365606 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359018    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359021    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359023    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359026    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359028    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359031    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359034    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359037    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359039    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359042    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359044    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359047    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359049    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359052    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359054    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359057    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359059    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359062    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359064    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.366088 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359067    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359071    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359074    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359077    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359079    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359082    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359084    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359087    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359089    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359094    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359097    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359099    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359102    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359104    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359107    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359109    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359112    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359115    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359117    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.366540 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359120    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359123    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359125    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359128    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359130    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359133    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359136    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359138    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359141    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359143    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359146    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359148    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359151    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359154    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359157    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359161    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359163    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359166    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.366999 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359169    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.359190    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359224    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359229    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359232    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359236    2581 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359239    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359241    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359244    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359248    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359251    2581 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359254    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359256    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359259    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359261    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359265    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:56:29.367423 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359268    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359271    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359273    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359276    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359278    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359281    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359285    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359289    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359292    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359295    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359298    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359300    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359303    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359306    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359308    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359311    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359313    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359316    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359318    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:56:29.367813 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359321    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359323    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359327    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359330    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359332    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359335    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359339    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359341    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359344    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359346    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359349    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359352    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359354    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359358    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359361    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359363    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359366    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359368    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359371    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:56:29.368257 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359375    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359379    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359382    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359385    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359388    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359391    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359393    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359396    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359398    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359401    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359403    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359405    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359408    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359410    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359413    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359417    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359419    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359422    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359425    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359429    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:56:29.368714 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359431    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359434    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359436    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359439    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359441    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359444    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359446    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359450    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359452    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359455    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359457    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359460    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359462    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359465    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359467    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359470    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359473    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359475    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359478    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359480    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:56:29.369212 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359483    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359485    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359488    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359490    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359493    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359495    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359498    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359503    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359505    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359508    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359510    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359514    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359517    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:29.359519    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.359568    2581 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.362966    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.363857    2581 server.go:1014] "Starting client certificate rotation"
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.363952    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 00:56:29.369816 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.363982    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 00:56:29.370252 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.365835    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 00:56:29.370252 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.366937    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 00:56:29.379171 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.379158    2581 log.go:25] "Validated CRI v1 runtime API"
Nov 04 00:56:29.382159 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.382140    2581 log.go:25] "Validated CRI v1 image API"
Nov 04 00:56:29.383985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.383958    2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 00:56:29.388549 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.388529    2581 fs.go:135] Filesystem UUIDs: map[01087df5-4860-4dad-912f-19ddf08b71a3:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 88f2c2f9-d576-47f4-b068-c02e9645b516:/dev/nvme0n1p4]
Nov 04 00:56:29.388606 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.388549    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 04 00:56:29.394816 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.394680    2581 manager.go:217] Machine: {Timestamp:2026-11-04 00:56:29.392217578 +0000 UTC m=+0.335413054 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099244 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ad30b0441188ffcf7a8733bddce3a SystemUUID:ec2ad30b-0441-188f-fcf7-a8733bddce3a BootID:c80544fb-1225-444e-ad04-22e8282c30b3 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:f0:d0:96:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:f0:d0:96:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:e8:8f:17:c1:3d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 00:56:29.395347 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.395337    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 00:56:29.395465 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.395453    2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 00:56:29.398185 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.398161    2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 00:56:29.398315 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.398186    2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-128.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 00:56:29.398363 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.398317    2581 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 00:56:29.398363 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.398325    2581 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 00:56:29.398363 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.398336    2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 00:56:29.398363 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.398349    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 00:56:29.399062 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.399052    2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 00:56:29.401562 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.401550    2581 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 00:56:29.401685 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.401676    2581 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 00:56:29.402253 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.402243    2581 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 00:56:29.402283 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.402257    2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 00:56:29.402283 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.402275    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 00:56:29.402357 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.402286    2581 kubelet.go:405] "Adding apiserver pod source"
Nov 04 00:56:29.402357 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.402301    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 00:56:29.402706 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.402694    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 00:56:29.404777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.404746    2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 00:56:29.412596 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.412580    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 00:56:29.414449 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414436    2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 00:56:29.414511 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414466    2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 00:56:29.414629 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414619    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 00:56:29.414662 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414635    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 00:56:29.414662 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414641    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 00:56:29.414662 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414647    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 00:56:29.414662 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414653    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 00:56:29.414662 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.414659    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 00:56:29.415563 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.415551    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 00:56:29.415563 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.415561    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 00:56:29.415647 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.415568    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 00:56:29.415647 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.415575    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 00:56:29.415647 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.415584    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 00:56:29.415647 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.415593    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 00:56:29.416420 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.416411    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 00:56:29.416420 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.416421    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 00:56:29.420149 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.420135    2581 server.go:1263] "Started kubelet"
Nov 04 00:56:29.420261 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.420234    2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 00:56:29.420341 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.420221    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 00:56:29.420372 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.420365    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 00:56:29.420866 ip-10-0-136-128 systemd[1]: Started Kubernetes Kubelet.
Nov 04 00:56:29.422158 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.422138    2581 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 00:56:29.422603 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.422586    2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 00:56:29.424862 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.424843    2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 04 00:56:29.428764 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.428746    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 00:56:29.429231 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.429217    2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 00:56:29.429332 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.429310    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:29.429840 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.429725    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 00:56:29.429840 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.429739    2581 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 00:56:29.429840 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.429761    2581 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 00:56:29.429840 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.429819    2581 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 00:56:29.429840 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.429830    2581 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 00:56:29.430059 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430046    2581 factory.go:55] Registering systemd factory
Nov 04 00:56:29.430091 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430063    2581 factory.go:223] Registration of the systemd container factory successfully
Nov 04 00:56:29.430268 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430256    2581 factory.go:153] Registering CRI-O factory
Nov 04 00:56:29.430268 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430268    2581 factory.go:223] Registration of the crio container factory successfully
Nov 04 00:56:29.430333 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430310    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 00:56:29.430333 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430323    2581 factory.go:103] Registering Raw factory
Nov 04 00:56:29.430333 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.430334    2581 manager.go:1199] Started watching for new ooms in manager
Nov 04 00:56:29.431328 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.431310    2581 manager.go:319] Starting recovery of all containers
Nov 04 00:56:29.443192 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.443172    2581 manager.go:324] Recovery completed
Nov 04 00:56:29.446199 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.445500    2581 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Nov 04 00:56:29.448521 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.448507    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.450430 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.450405    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.450511 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.450443    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.450511 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.450454    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.450997 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.450976    2581 cpu_manager.go:226] "Starting" policy="none"
Nov 04 00:56:29.450997 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.450990    2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 00:56:29.451125 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.451010    2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 00:56:29.451125 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.451024    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.451919 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.451900    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.452028 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.451930    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.452028 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.451940    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.453402 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.453389    2581 policy_none.go:50] "Start"
Nov 04 00:56:29.453453 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.453404    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 00:56:29.453453 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.453414    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 00:56:29.454568 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.454556    2581 policy_none.go:44] "Start"
Nov 04 00:56:29.492362 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.492314    2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 00:56:29.494956 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.494942    2581 manager.go:341] "Starting Device Plugin manager"
Nov 04 00:56:29.495050 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.494981    2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 00:56:29.495050 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.494993    2581 server.go:87] "Starting device plugin registration server"
Nov 04 00:56:29.495270 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.495259    2581 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 00:56:29.495310 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.495273    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 00:56:29.495527 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.495375    2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 00:56:29.495527 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.495470    2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 00:56:29.495527 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.495479    2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 00:56:29.495976 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.495951    2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 00:56:29.496043 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.495998    2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:29.546906 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.546868    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-128.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 04 00:56:29.546906 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.546867    2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-128.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 04 00:56:29.547753 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.546866    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-128.us-west-2.compute.internal.18e4b19158e58007  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-128.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-128.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 00:56:29.420109831 +0000 UTC m=+0.363305307,LastTimestamp:2026-11-04 00:56:29.420109831 +0000 UTC m=+0.363305307,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-128.us-west-2.compute.internal,}"
Nov 04 00:56:29.574535 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.574511    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 00:56:29.575918 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.575898    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 00:56:29.575918 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.575921    2581 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 00:56:29.576054 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.575943    2581 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 00:56:29.576054 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.575978    2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 04 00:56:29.595565 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.595497    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.596318 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.596301    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.596410 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.596334    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.596410 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.596350    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.596410 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.596380    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.612923 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.612895    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 04 00:56:29.613034 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.612916    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-128.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 04 00:56:29.613034 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.612843    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-128.us-west-2.compute.internal.18e4b1915ab4207f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-128.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-128.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-128.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 00:56:29.450428543 +0000 UTC m=+0.393624020,LastTimestamp:2026-11-04 00:56:29.450428543 +0000 UTC m=+0.393624020,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-128.us-west-2.compute.internal,}"
Nov 04 00:56:29.613034 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.612996    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 04 00:56:29.644006 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.643988    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 00:56:29.644118 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.644101    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m2lkp"
Nov 04 00:56:29.644159 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.644123    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m2lkp"
Nov 04 00:56:29.677047 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.677017    2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"]
Nov 04 00:56:29.677136 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.677085    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.683596 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.683576    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.683668 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.683602    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.683668 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.683614    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.684897 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.684885    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.685036 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685023    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.685074 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685056    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.685685 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685667    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.685774 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685693    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.685774 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685703    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.685774 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685703    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.685774 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685722    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.685774 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.685753    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.687001 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.686988    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.687046 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.687025    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:29.687724 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.687708    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:29.687813 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.687747    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:29.687813 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.687760    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:29.706109 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.706093    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.709812 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.709795    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.716413 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.716397    2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.716482 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.716417    2581 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-128.us-west-2.compute.internal\": node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:29.731200 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.731182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.731296 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.731235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.731296 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.731279    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d4b0eb194eb22fd85d7942f84eead7e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"1d4b0eb194eb22fd85d7942f84eead7e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.784547 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.784523    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:56:29.832187 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.832165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.832259 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.832194    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.832259 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.832219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d4b0eb194eb22fd85d7942f84eead7e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"1d4b0eb194eb22fd85d7942f84eead7e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.832328 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.832258    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d4b0eb194eb22fd85d7942f84eead7e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"1d4b0eb194eb22fd85d7942f84eead7e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.832328 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.832260    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.832328 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:29.832264    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:29.837126 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.837112    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:29.937696 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:29.937626    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:30.009427 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:30.009405    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:30.013339 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:30.013062    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:30.038064 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:30.038042    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:30.138955 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:30.138928    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:30.239562 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:30.239504    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:31.751281 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:31.751262    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:31.775434 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:31.775412    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 04 00:56:31.775848 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:31.775505    2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 04 00:56:31.775848 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:31.775573    2581 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ab4f6215569114d8cb596808b4941cb0-c1f4c936e9dcd2c2.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-136-128.us-west-2.compute.internal.18e4b1915ab46ca9\": read tcp 10.0.136.128:54636->3.131.145.49:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-136-128.us-west-2.compute.internal.18e4b1915ab46ca9  default   5235 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-128.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-128.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-128.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 00:56:29 +0000 UTC,LastTimestamp:2026-11-04 00:56:29.683607139 +0000 UTC m=+0.626802614,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-128.us-west-2.compute.internal,}"
Nov 04 00:56:31.852297 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:31.852272    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:31.952961 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:31.952904    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.053316 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.053282    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.058958 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.058924    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 00:51:31 +0000 UTC" deadline="2028-08-09 15:46:48.378647772 +0000 UTC"
Nov 04 00:56:32.058958 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.058956    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15470h50m16.319694845s"
Nov 04 00:56:32.116318 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.116295    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:56:32.133445 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.133425    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:56:32.134387 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.134366    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:56:32.134490 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.134404    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:56:32.134490 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.134420    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:56:32.153961 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.153941    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.254959 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.254893    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.355546 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.355521    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.456101 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.456069    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.545230 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.545172    2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:56:32.556147 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.556127    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:56:32.582809 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:32.582779    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod929e66bb82351eba2f21f01154393561.slice/crio-81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af WatchSource:0}: Error finding container 81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af: Status 404 returned error can't find the container with id 81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af
Nov 04 00:56:32.583019 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:32.583004    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4b0eb194eb22fd85d7942f84eead7e.slice/crio-b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4 WatchSource:0}: Error finding container b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4: Status 404 returned error can't find the container with id b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4
Nov 04 00:56:32.588718 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.588703    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 00:56:32.623558 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.623538    2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:56:32.640741 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.640711    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:32.680595 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.680571    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:56:32.700202 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.700183    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 00:56:32.701214 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.701201    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:56:32.758390 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.758368    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 00:56:32.814468 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.814401    2581 apiserver.go:52] "Watching apiserver"
Nov 04 00:56:32.840573 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.840550    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 04 00:56:32.917539 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.917514    2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 00:56:32.917803 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.917787    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-456n2","openshift-multus/multus-htt4g","openshift-multus/network-metrics-daemon-7vlfq","openshift-network-operator/iptables-alerter-jv4jd","openshift-image-registry/node-ca-6sgf4","openshift-network-diagnostics/network-check-target-frjrl","openshift-ovn-kubernetes/ovnkube-node-w557m","openshift-dns/node-resolver-rc99q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27","openshift-cluster-node-tuning-operator/tuned-wrcbh","openshift-multus/multus-additional-cni-plugins-zb7k2","kube-system/global-pull-secret-syncer-rcwm8"]
Nov 04 00:56:32.921685 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.921653    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.923953 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.923931    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.926112 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.926096    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:32.926193 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.926150    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:32.928376 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.928362    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:32.930692 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.930677    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:32.933477 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.933456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:32.933548 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.933514    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:32.935944 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.935928    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:32.936012 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:32.935976    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:32.938344 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.938330    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:32.940881 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.940865    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.943123 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.943107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:32.945536 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.945519    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:32.948222 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.948201    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:32.952951 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.952932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.953039 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.952962    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm766\" (UniqueName: \"kubernetes.io/projected/b2fa4fe0-3667-44c4-963c-4572a330d29c-kube-api-access-wm766\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.953039 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.952985    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-log-socket\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953039 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de508010-fa70-4f37-a691-bdecaecdd3e7-konnectivity-ca\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:32.953177 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-serviceca\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:32.953177 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953109    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-dbus\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:32.953177 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953151    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:32.953322 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953176    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-hostroot\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953322 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953224    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de508010-fa70-4f37-a691-bdecaecdd3e7-agent-certs\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:32.953322 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.953322 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-netns\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953464 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953362    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4813a1fc-7a03-487f-b9fa-8d1a7250768f-iptables-alerter-script\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:32.953464 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-k8s-cni-cncf-io\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953464 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953436    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953464 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953460    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:32.953581 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953486    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qvnz\" (UniqueName: \"kubernetes.io/projected/c5dca767-45ed-4760-9d96-d2db34951608-kube-api-access-6qvnz\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:32.953581 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-etc-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953581 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953532    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-env-overrides\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953581 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953560    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dbxh\" (UniqueName: \"kubernetes.io/projected/ef3c5a66-a065-45ac-8979-a13b2e382041-kube-api-access-5dbxh\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953723 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953611    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-kubelet\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953723 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953647    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4813a1fc-7a03-487f-b9fa-8d1a7250768f-host-slash\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:32.953723 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-multus-certs\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953723 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953709    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-etc-kubernetes\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953732    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/352db7a3-daed-4f66-81db-7054c082d043-hosts-file\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953794    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-bin\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953825    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf2sc\" (UniqueName: \"kubernetes.io/projected/352db7a3-daed-4f66-81db-7054c082d043-kube-api-access-bf2sc\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953851    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953878    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-cni-binary-copy\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953901    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-kubelet\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953922    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-script-lib\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.953985 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.953971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-system-cni-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954038    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrp79\" (UniqueName: \"kubernetes.io/projected/4813a1fc-7a03-487f-b9fa-8d1a7250768f-kube-api-access-wrp79\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954074    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-conf-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-daemon-config\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-netd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954131    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-systemd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-var-lib-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954201    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-config\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954248    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-socket-dir-parent\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954264    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-netns\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954298 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-bin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954307    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-system-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954325    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-ovn\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954340    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-cnibin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954353    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-multus\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85125fc3-99ec-427f-84a5-910d207e22cd-ovn-node-metrics-cert\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954407    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gggw2\" (UniqueName: \"kubernetes.io/projected/85125fc3-99ec-427f-84a5-910d207e22cd-kube-api-access-gggw2\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954440    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-cnibin\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954470    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-systemd-units\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954497    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-host\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954513    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6xz\" (UniqueName: \"kubernetes.io/projected/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-kube-api-access-qt6xz\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954534    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352db7a3-daed-4f66-81db-7054c082d043-tmp-dir\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-kubelet-config\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954618    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:32.954708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954652    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-node-log\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.955157 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954727    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:32.955157 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954743    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:32.955157 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.954762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-slash\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:32.974490 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.974463    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 00:56:32.974490 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.974482    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 00:56:32.974601 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.974521    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4d9rj\""
Nov 04 00:56:32.974635 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.974627    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 00:56:32.974851 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.974827    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 00:56:32.974950 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.974893    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 00:56:32.975144 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.975130    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 00:56:32.978151 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.978135    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 00:56:32.990887 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.990849    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"1d4b0eb194eb22fd85d7942f84eead7e","Type":"ContainerStarted","Data":"b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4"}
Nov 04 00:56:32.991768 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:32.991741    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"929e66bb82351eba2f21f01154393561","Type":"ContainerStarted","Data":"81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af"}
Nov 04 00:56:33.020869 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.020851    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 04 00:56:33.020952 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.020887    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5grms"
Nov 04 00:56:33.041550 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.041534    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 00:56:33.055459 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.055535 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm766\" (UniqueName: \"kubernetes.io/projected/b2fa4fe0-3667-44c4-963c-4572a330d29c-kube-api-access-wm766\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.055535 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055482    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-log-socket\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.055535 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de508010-fa70-4f37-a691-bdecaecdd3e7-konnectivity-ca\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:33.055535 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055525    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-registration-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.055712 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055549    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.055712 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055576    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-log-socket\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.055712 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055616    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmbpl\" (UniqueName: \"kubernetes.io/projected/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kube-api-access-pmbpl\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.055712 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-run\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.055712 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055711    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-serviceca\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.055855 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-dbus\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:33.055855 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055777    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:33.055855 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055790    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-hostroot\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.055855 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055804    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de508010-fa70-4f37-a691-bdecaecdd3e7-agent-certs\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:33.055855 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055820    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25ppk\" (UniqueName: \"kubernetes.io/projected/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-kube-api-access-25ppk\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.055855 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055865    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-hostroot\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.055901    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-netns\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055947    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4813a1fc-7a03-487f-b9fa-8d1a7250768f-iptables-alerter-script\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.055959    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret podName:d41414c4-11d2-4e45-aa39-8f33726a40fd nodeName:}" failed. No retries permitted until 2026-11-04 00:56:33.555943288 +0000 UTC m=+3.088204554 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret") pod "global-pull-secret-syncer-rcwm8" (UID: "d41414c4-11d2-4e45-aa39-8f33726a40fd") : object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.055971    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056009    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-k8s-cni-cncf-io\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056030    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-netns\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056037    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-k8s-cni-cncf-io\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056050    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-dbus\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056048    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056070    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056096    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qvnz\" (UniqueName: \"kubernetes.io/projected/c5dca767-45ed-4760-9d96-d2db34951608-kube-api-access-6qvnz\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056137    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-sys-fs\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-etc-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056180    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-env-overrides\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-modprobe-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.056242    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056255    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dbxh\" (UniqueName: \"kubernetes.io/projected/ef3c5a66-a065-45ac-8979-a13b2e382041-kube-api-access-5dbxh\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.056281    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:33.556269223 +0000 UTC m=+3.088530482 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056288    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-etc-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056294    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-var-lib-kubelet\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056393    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-tuned\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056425    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-kubelet\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4813a1fc-7a03-487f-b9fa-8d1a7250768f-host-slash\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056463    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-multus-certs\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056490    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-multus-certs\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056492    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-kubelet\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.056608 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056491    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-etc-kubernetes\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056509    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4813a1fc-7a03-487f-b9fa-8d1a7250768f-host-slash\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056517    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-etc-kubernetes\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/352db7a3-daed-4f66-81db-7054c082d043-hosts-file\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056646    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/352db7a3-daed-4f66-81db-7054c082d043-hosts-file\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056689    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-lib-modules\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056707    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-bin\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056721    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf2sc\" (UniqueName: \"kubernetes.io/projected/352db7a3-daed-4f66-81db-7054c082d043-kube-api-access-bf2sc\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056735    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-env-overrides\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056735    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysconfig\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056773    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-bin\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056798    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-cni-binary-copy\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-kubelet\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056897    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-kubelet\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.057405 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056574    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.056903    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-systemd\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057365    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-host\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057388    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-tmp\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057411    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-script-lib\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057433    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-system-cni-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057469    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrp79\" (UniqueName: \"kubernetes.io/projected/4813a1fc-7a03-487f-b9fa-8d1a7250768f-kube-api-access-wrp79\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-conf-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057541    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-daemon-config\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057548    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-system-cni-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-netd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057581    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-netd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-device-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057603    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-conf-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-systemd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-var-lib-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057687    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-systemd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058049 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057724    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-config\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057753    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-var-lib-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057762    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-socket-dir-parent\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057791    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-netns\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057806    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-socket-dir-parent\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-bin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057831    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-bin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057841    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-netns\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057851    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-system-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057873    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-ovn\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-cnibin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057932    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-socket-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057938    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-ovn\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057956    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-script-lib\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057966    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-system-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057957    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.058757 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057988    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-cnibin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.057996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-multus\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85125fc3-99ec-427f-84a5-910d207e22cd-ovn-node-metrics-cert\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058027    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gggw2\" (UniqueName: \"kubernetes.io/projected/85125fc3-99ec-427f-84a5-910d207e22cd-kube-api-access-gggw2\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058052    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-conf\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058059    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-multus\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058075    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-cnibin\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058092    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-config\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-systemd-units\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058128    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-cnibin\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-host\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058149    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-systemd-units\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058154    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6xz\" (UniqueName: \"kubernetes.io/projected/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-kube-api-access-qt6xz\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058161    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-host\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058178    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352db7a3-daed-4f66-81db-7054c082d043-tmp-dir\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058204    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-kubernetes\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-sys\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.059255 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058306    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-kubelet-config\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058330    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-kubelet-config\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058352    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058368    2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058385    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-node-log\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058405    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058420    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058435    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352db7a3-daed-4f66-81db-7054c082d043-tmp-dir\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-slash\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058482    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058511    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-slash\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.059734 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.058537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-node-log\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.061711 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.061694    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85125fc3-99ec-427f-84a5-910d207e22cd-ovn-node-metrics-cert\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.074759 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.074714    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gczmn\""
Nov 04 00:56:33.075344 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075324    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 00:56:33.075461 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075373    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ql7fx\""
Nov 04 00:56:33.075461 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075412    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tdv96\""
Nov 04 00:56:33.075461 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075443    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 00:56:33.075461 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075451    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 00:56:33.075461 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075428    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075458    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075495    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-66h22\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075510    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075525    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075558    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075563    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075565    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075577    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 00:56:33.075713 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075576    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 00:56:33.076276 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075833    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 00:56:33.076276 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075843    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 00:56:33.076276 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075884    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-qxmlz\""
Nov 04 00:56:33.076276 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075888    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6tf45\""
Nov 04 00:56:33.076276 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.075941    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 00:56:33.076494 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076320    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 00:56:33.076632 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076614    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 00:56:33.076694 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076643    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 00:56:33.076694 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076643    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 00:56:33.076694 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076644    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 00:56:33.076866 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076841    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de508010-fa70-4f37-a691-bdecaecdd3e7-konnectivity-ca\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:33.076932 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076915    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6gr6q\""
Nov 04 00:56:33.076982 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076937    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2w2b9\""
Nov 04 00:56:33.076982 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.076939    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-serviceca\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.077546 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.077526    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-cni-binary-copy\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.078910 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.077914    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.078910 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.078338    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-daemon-config\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.080006 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.079988    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.086517 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.086499    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4813a1fc-7a03-487f-b9fa-8d1a7250768f-iptables-alerter-script\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.086581 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.086565    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.088026 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.088005    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de508010-fa70-4f37-a691-bdecaecdd3e7-agent-certs\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:33.112221 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.112192    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm766\" (UniqueName: \"kubernetes.io/projected/b2fa4fe0-3667-44c4-963c-4572a330d29c-kube-api-access-wm766\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.112328 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.112253    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 00:56:33.112328 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.112273    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 00:56:33.112328 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.112287    2581 projected.go:196] Error preparing data for projected volume kube-api-access-7l7cg for pod openshift-network-diagnostics/network-check-target-frjrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:33.112441 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.112340    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg podName:728c5532-b88d-4ccc-bb31-a0278f5f84f8 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:33.612325782 +0000 UTC m=+3.144587060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7l7cg" (UniqueName: "kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg") pod "network-check-target-frjrl" (UID: "728c5532-b88d-4ccc-bb31-a0278f5f84f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:33.112926 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.112902    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf2sc\" (UniqueName: \"kubernetes.io/projected/352db7a3-daed-4f66-81db-7054c082d043-kube-api-access-bf2sc\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.113010 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.112946    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dbxh\" (UniqueName: \"kubernetes.io/projected/ef3c5a66-a065-45ac-8979-a13b2e382041-kube-api-access-5dbxh\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.114063 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.114038    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qvnz\" (UniqueName: \"kubernetes.io/projected/c5dca767-45ed-4760-9d96-d2db34951608-kube-api-access-6qvnz\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:33.114063 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.114059    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrp79\" (UniqueName: \"kubernetes.io/projected/4813a1fc-7a03-487f-b9fa-8d1a7250768f-kube-api-access-wrp79\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.114172 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.114155    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6xz\" (UniqueName: \"kubernetes.io/projected/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-kube-api-access-qt6xz\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.114290 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.114275    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gggw2\" (UniqueName: \"kubernetes.io/projected/85125fc3-99ec-427f-84a5-910d207e22cd-kube-api-access-gggw2\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.159097 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-sys-fs\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159177 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-modprobe-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159177 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159119    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-var-lib-kubelet\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159178    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-var-lib-kubelet\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159184    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-sys-fs\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-tuned\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-lib-modules\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159247    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysconfig\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159259    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-modprobe-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159284 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159270    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-systemd\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159309    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-host\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159325    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysconfig\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-tmp\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159373    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-device-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159382    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-systemd\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159393    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-host\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159409    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-socket-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159375    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-lib-modules\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159432    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159439    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-device-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159450    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159513    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159511    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-conf\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159531    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-socket-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159553    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-kubernetes\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.159600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159576    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-sys\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159617    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-registration-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159638    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-conf\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159681    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-sys\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159682    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-kubernetes\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmbpl\" (UniqueName: \"kubernetes.io/projected/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kube-api-access-pmbpl\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159707    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159726    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-registration-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159814    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-run\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159883    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25ppk\" (UniqueName: \"kubernetes.io/projected/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-kube-api-access-25ppk\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.160227 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.159908    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-run\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.161275 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.161258    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-tuned\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.161333 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.161286    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-tmp\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.215999 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.215976    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25ppk\" (UniqueName: \"kubernetes.io/projected/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-kube-api-access-25ppk\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.216110 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.216092    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmbpl\" (UniqueName: \"kubernetes.io/projected/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kube-api-access-pmbpl\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.251045 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.251027    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:56:33.259943 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.259926    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:56:33.260830 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.260808    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4813a1fc_7a03_487f_b9fa_8d1a7250768f.slice/crio-c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce WatchSource:0}: Error finding container c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce: Status 404 returned error can't find the container with id c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce
Nov 04 00:56:33.262587 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.262570    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:33.265280 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.265264    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:33.267610 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.267555    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-htt4g"
Nov 04 00:56:33.270466 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.270448    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f018fcc_7d38_494c_bfe5_67dc725ec5a7.slice/crio-fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc WatchSource:0}: Error finding container fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc: Status 404 returned error can't find the container with id fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc
Nov 04 00:56:33.271233 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.271214    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:56:33.271943 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.271921    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85125fc3_99ec_427f_84a5_910d207e22cd.slice/crio-71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48 WatchSource:0}: Error finding container 71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48: Status 404 returned error can't find the container with id 71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48
Nov 04 00:56:33.273416 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.273394    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:56:33.275790 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.275772    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde508010_fa70_4f37_a691_bdecaecdd3e7.slice/crio-4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f WatchSource:0}: Error finding container 4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f: Status 404 returned error can't find the container with id 4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f
Nov 04 00:56:33.278931 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.278901    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef3c5a66_a065_45ac_8979_a13b2e382041.slice/crio-acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923 WatchSource:0}: Error finding container acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923: Status 404 returned error can't find the container with id acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923
Nov 04 00:56:33.280199 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.280183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:56:33.282181 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.282166    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:56:33.282470 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.282451    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2fa4fe0_3667_44c4_963c_4572a330d29c.slice/crio-a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584 WatchSource:0}: Error finding container a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584: Status 404 returned error can't find the container with id a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584
Nov 04 00:56:33.285053 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.285034    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod352db7a3_daed_4f66_81db_7054c082d043.slice/crio-7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd WatchSource:0}: Error finding container 7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd: Status 404 returned error can't find the container with id 7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd
Nov 04 00:56:33.291125 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.291107    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfcc3b0_dbd0_42b0_b1f2_60f7d3331163.slice/crio-8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275 WatchSource:0}: Error finding container 8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275: Status 404 returned error can't find the container with id 8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275
Nov 04 00:56:33.293328 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:56:33.293311    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebd2dcd_06f3_4aa9_83cd_f3a0a9bd7ccb.slice/crio-a9642eca8b602c8790986f7b2ef5946f341090fc64bd164b6b687e557e3a811d WatchSource:0}: Error finding container a9642eca8b602c8790986f7b2ef5946f341090fc64bd164b6b687e557e3a811d: Status 404 returned error can't find the container with id a9642eca8b602c8790986f7b2ef5946f341090fc64bd164b6b687e557e3a811d
Nov 04 00:56:33.563417 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.563343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:33.563417 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.563379    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:33.563598 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.563497    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:33.563598 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.563552    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret podName:d41414c4-11d2-4e45-aa39-8f33726a40fd nodeName:}" failed. No retries permitted until 2026-11-04 00:56:34.563539161 +0000 UTC m=+4.095800411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret") pod "global-pull-secret-syncer-rcwm8" (UID: "d41414c4-11d2-4e45-aa39-8f33726a40fd") : object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:33.563598 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.563497    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:33.563598 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.563587    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:34.563577681 +0000 UTC m=+4.095838931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:33.664280 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.664242    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:33.664446 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.664386    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 00:56:33.664446 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.664409    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 00:56:33.664446 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.664418    2581 projected.go:196] Error preparing data for projected volume kube-api-access-7l7cg for pod openshift-network-diagnostics/network-check-target-frjrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:33.664556 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:33.664480    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg podName:728c5532-b88d-4ccc-bb31-a0278f5f84f8 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:34.66445811 +0000 UTC m=+4.196719357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l7cg" (UniqueName: "kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg") pod "network-check-target-frjrl" (UID: "728c5532-b88d-4ccc-bb31-a0278f5f84f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:33.994748 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.994647    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wrcbh" event={"ID":"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb","Type":"ContainerStarted","Data":"a9642eca8b602c8790986f7b2ef5946f341090fc64bd164b6b687e557e3a811d"}
Nov 04 00:56:33.996267 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.996239    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerStarted","Data":"a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584"}
Nov 04 00:56:33.997625 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.997590    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jv4jd" event={"ID":"4813a1fc-7a03-487f-b9fa-8d1a7250768f","Type":"ContainerStarted","Data":"c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce"}
Nov 04 00:56:33.999272 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:33.999247    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27" event={"ID":"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163","Type":"ContainerStarted","Data":"8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275"}
Nov 04 00:56:34.000873 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.000852    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rc99q" event={"ID":"352db7a3-daed-4f66-81db-7054c082d043","Type":"ContainerStarted","Data":"7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd"}
Nov 04 00:56:34.002705 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.002275    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-htt4g" event={"ID":"ef3c5a66-a065-45ac-8979-a13b2e382041","Type":"ContainerStarted","Data":"acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923"}
Nov 04 00:56:34.003418 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.003396    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-456n2" event={"ID":"de508010-fa70-4f37-a691-bdecaecdd3e7","Type":"ContainerStarted","Data":"4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f"}
Nov 04 00:56:34.005334 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.005213    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48"}
Nov 04 00:56:34.006478 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.006456    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6sgf4" event={"ID":"5f018fcc-7d38-494c-bfe5-67dc725ec5a7","Type":"ContainerStarted","Data":"fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc"}
Nov 04 00:56:34.023752 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.023719    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 00:51:32 +0000 UTC" deadline="2028-03-29 16:31:50.896836548 +0000 UTC"
Nov 04 00:56:34.023856 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.023754    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12279h35m16.873087568s"
Nov 04 00:56:34.571835 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.571793    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:34.572030 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.571856    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:34.572104 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.572045    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:34.572159 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.572115    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:36.572095529 +0000 UTC m=+6.104356794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:34.572262 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.572246    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:34.572316 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.572292    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret podName:d41414c4-11d2-4e45-aa39-8f33726a40fd nodeName:}" failed. No retries permitted until 2026-11-04 00:56:36.572280373 +0000 UTC m=+6.104541634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret") pod "global-pull-secret-syncer-rcwm8" (UID: "d41414c4-11d2-4e45-aa39-8f33726a40fd") : object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:34.673418 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.673118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:34.673418 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.673291    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 00:56:34.673418 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.673308    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 00:56:34.673418 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.673321    2581 projected.go:196] Error preparing data for projected volume kube-api-access-7l7cg for pod openshift-network-diagnostics/network-check-target-frjrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:34.673418 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.673377    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg podName:728c5532-b88d-4ccc-bb31-a0278f5f84f8 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:36.673360868 +0000 UTC m=+6.205622120 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l7cg" (UniqueName: "kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg") pod "network-check-target-frjrl" (UID: "728c5532-b88d-4ccc-bb31-a0278f5f84f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:34.990819 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.990583    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:34.990819 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.990723    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:34.990819 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.990811    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:34.991081 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.990902    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:34.991081 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:34.990959    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:34.991081 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:34.991045    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:35.024606 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:35.024538    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 00:51:32 +0000 UTC" deadline="2028-04-18 14:14:00.565937351 +0000 UTC"
Nov 04 00:56:35.024606 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:35.024597    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12757h17m25.541344645s"
Nov 04 00:56:36.590444 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:36.590139    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:36.590444 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:36.590410    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:36.590444 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.590415    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:36.591113 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.590489    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret podName:d41414c4-11d2-4e45-aa39-8f33726a40fd nodeName:}" failed. No retries permitted until 2026-11-04 00:56:40.590470888 +0000 UTC m=+10.122732150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret") pod "global-pull-secret-syncer-rcwm8" (UID: "d41414c4-11d2-4e45-aa39-8f33726a40fd") : object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:36.591113 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.590553    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:36.591113 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.590605    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:40.590588312 +0000 UTC m=+10.122849574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:36.690990 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:36.690955    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:36.691180 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.691165    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 00:56:36.691306 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.691193    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 00:56:36.691306 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.691206    2581 projected.go:196] Error preparing data for projected volume kube-api-access-7l7cg for pod openshift-network-diagnostics/network-check-target-frjrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:36.691306 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.691262    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg podName:728c5532-b88d-4ccc-bb31-a0278f5f84f8 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:40.691244996 +0000 UTC m=+10.223506244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l7cg" (UniqueName: "kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg") pod "network-check-target-frjrl" (UID: "728c5532-b88d-4ccc-bb31-a0278f5f84f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:36.989743 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:36.989544    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:36.989743 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.989724    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:36.989968 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:36.989791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:36.989968 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.989878    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:36.989968 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:36.989918    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:36.990071 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:36.989983    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:37.029682 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:37.029636    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="1ba66e20db540e81ba625445af1dd32fd67d2c405348baae4b69fc3b0ddca395" exitCode=0
Nov 04 00:56:37.029831 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:37.029734    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerDied","Data":"1ba66e20db540e81ba625445af1dd32fd67d2c405348baae4b69fc3b0ddca395"}
Nov 04 00:56:37.040409 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:37.039994    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"1d4b0eb194eb22fd85d7942f84eead7e","Type":"ContainerStarted","Data":"3c3bb287493162c8f65080df6553f29b14fc2777717ff30e357133f2f792bb99"}
Nov 04 00:56:37.050909 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:37.050884    2581 generic.go:356] "Generic (PLEG): container finished" podID="929e66bb82351eba2f21f01154393561" containerID="27107775c6b7175b71bcafbc595148fe6b9af505a587f8fbf300de6ccd5a8128" exitCode=0
Nov 04 00:56:37.051035 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:37.050927    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"929e66bb82351eba2f21f01154393561","Type":"ContainerDied","Data":"27107775c6b7175b71bcafbc595148fe6b9af505a587f8fbf300de6ccd5a8128"}
Nov 04 00:56:37.475454 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:37.475335    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal" podStartSLOduration=5.475315176 podStartE2EDuration="5.475315176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:37.474157575 +0000 UTC m=+7.006418844" watchObservedRunningTime="2026-11-04 00:56:37.475315176 +0000 UTC m=+7.007576449"
Nov 04 00:56:38.073382 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:38.073346    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"929e66bb82351eba2f21f01154393561","Type":"ContainerStarted","Data":"fe5da2a38472787b10f23281d280d8fb5364914dc23048cee73729e7f5d6138b"}
Nov 04 00:56:38.989456 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:38.989398    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:38.989618 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:38.989522    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:38.989618 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:38.989590    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:38.989767 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:38.989658    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:38.989767 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:38.989731    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:38.989868 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:38.989815    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:39.079768 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:39.079738    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-456n2" event={"ID":"de508010-fa70-4f37-a691-bdecaecdd3e7","Type":"ContainerStarted","Data":"d67a74d1580a02ac774b96cd895128805c9b0d982bb02de399b30e26eb077c10"}
Nov 04 00:56:39.193183 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:39.193115    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" podStartSLOduration=7.193099023 podStartE2EDuration="7.193099023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:38.185961707 +0000 UTC m=+7.718222977" watchObservedRunningTime="2026-11-04 00:56:39.193099023 +0000 UTC m=+8.725360291"
Nov 04 00:56:39.193618 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:39.193524    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-456n2" podStartSLOduration=3.488007461 podStartE2EDuration="8.193493173s" totalImagesPullingTime="4.705485712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:39.19308551 +0000 UTC m=+8.725346782" watchObservedRunningTime="2026-11-04 00:56:39.193493173 +0000 UTC m=+8.725754444"
Nov 04 00:56:40.083782 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.083657    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27" event={"ID":"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163","Type":"ContainerStarted","Data":"b04a2c5555468bcf0e27d31918ff4e42bb95ac04cd8c82e8ec2bc667fa97c9eb"}
Nov 04 00:56:40.089094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.089010    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6sgf4" event={"ID":"5f018fcc-7d38-494c-bfe5-67dc725ec5a7","Type":"ContainerStarted","Data":"5800700ac0c502319a9871c1ea369e58fe20b0411ce686822342c3d76640f92b"}
Nov 04 00:56:40.628896 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.628854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:40.629093 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.628914    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:40.629093 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.629034    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:40.629093 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.629080    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:48.629067236 +0000 UTC m=+18.161328496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:40.629245 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.629120    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:40.629245 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.629153    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret podName:d41414c4-11d2-4e45-aa39-8f33726a40fd nodeName:}" failed. No retries permitted until 2026-11-04 00:56:48.629141567 +0000 UTC m=+18.161402832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret") pod "global-pull-secret-syncer-rcwm8" (UID: "d41414c4-11d2-4e45-aa39-8f33726a40fd") : object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:40.730272 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.730233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:40.730449 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.730386    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 00:56:40.730449 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.730408    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 00:56:40.730449 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.730421    2581 projected.go:196] Error preparing data for projected volume kube-api-access-7l7cg for pod openshift-network-diagnostics/network-check-target-frjrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:40.730619 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.730475    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg podName:728c5532-b88d-4ccc-bb31-a0278f5f84f8 nodeName:}" failed. No retries permitted until 2026-11-04 00:56:48.730457829 +0000 UTC m=+18.262719097 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l7cg" (UniqueName: "kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg") pod "network-check-target-frjrl" (UID: "728c5532-b88d-4ccc-bb31-a0278f5f84f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:40.990379 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.990307    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:40.990510 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.990445    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:40.990510 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:40.990307    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:40.990568 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.990530    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:40.990568 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.990441    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:40.990630 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:40.990616    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:41.353975 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:41.353870    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:41.355411 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:41.355389    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:41.470606 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:41.470551    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6sgf4" podStartSLOduration=4.469739908 podStartE2EDuration="10.470538316s" totalImagesPullingTime="6.000798408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:40.201565345 +0000 UTC m=+9.733826614" watchObservedRunningTime="2026-11-04 00:56:41.470538316 +0000 UTC m=+11.002799584"
Nov 04 00:56:42.096948 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.096905    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rc99q" event={"ID":"352db7a3-daed-4f66-81db-7054c082d043","Type":"ContainerStarted","Data":"9076c7057ae38b3838fb76af3f11b7953c8faa06621da1793056a6a78007c1ff"}
Nov 04 00:56:42.099344 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.099315    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-htt4g" event={"ID":"ef3c5a66-a065-45ac-8979-a13b2e382041","Type":"ContainerStarted","Data":"f5df72d77d8a2a37114668d4e6f22316a2fcdb124171030173e3880bb3f5744d"}
Nov 04 00:56:42.101583 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.101553    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerStarted","Data":"0a785a25f01ed2d41991956c6389fe105c232344e9d2f64d21fabb1793c8e522"}
Nov 04 00:56:42.101944 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.101922    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:42.102464 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.102445    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:56:42.210155 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.210079    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-rc99q" podStartSLOduration=2.766601 podStartE2EDuration="11.210061173s" totalImagesPullingTime="8.443460173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:42.209829447 +0000 UTC m=+11.742090713" watchObservedRunningTime="2026-11-04 00:56:42.210061173 +0000 UTC m=+11.742322444"
Nov 04 00:56:42.262852 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.262607    2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 00:56:42.325839 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.325780    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-htt4g" podStartSLOduration=2.7309507 podStartE2EDuration="11.325761262s" totalImagesPullingTime="8.594810562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:42.324413866 +0000 UTC m=+11.856675137" watchObservedRunningTime="2026-11-04 00:56:42.325761262 +0000 UTC m=+11.858022533"
Nov 04 00:56:42.918301 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.918199    2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T00:56:42.262827676Z","UUID":"5b395af7-813b-4d25-bd73-adc9a449a053","Handler":null,"Name":"","Endpoint":""}
Nov 04 00:56:42.920095 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.920063    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 00:56:42.920095 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.920101    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 00:56:42.988024 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.987993    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:42.988192 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:42.988136    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:42.988497 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.988478    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:42.988608 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:42.988581    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:42.988742 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:42.988647    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:42.988742 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:42.988735    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:43.105185 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:43.105152    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="0a785a25f01ed2d41991956c6389fe105c232344e9d2f64d21fabb1793c8e522" exitCode=0
Nov 04 00:56:43.105344 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:43.105231    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerDied","Data":"0a785a25f01ed2d41991956c6389fe105c232344e9d2f64d21fabb1793c8e522"}
Nov 04 00:56:43.108122 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:43.108060    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27" event={"ID":"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163","Type":"ContainerStarted","Data":"db6cd387686ef24b67ee2fbfa59c18cb085153a821d8f71ee6c35767cd3aad5a"}
Nov 04 00:56:44.111243 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:44.111208    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jv4jd" event={"ID":"4813a1fc-7a03-487f-b9fa-8d1a7250768f","Type":"ContainerStarted","Data":"33c78705ece61884092e4e6734f6a621eaac81f18bb67006eca05ee256726642"}
Nov 04 00:56:44.224112 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:44.224049    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jv4jd" podStartSLOduration=4.478606347 podStartE2EDuration="13.22403133s" totalImagesPullingTime="8.745424983s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:44.223586277 +0000 UTC m=+13.755847549" watchObservedRunningTime="2026-11-04 00:56:44.22403133 +0000 UTC m=+13.756292599"
Nov 04 00:56:44.987611 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:44.987584    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:44.987751 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:44.987729    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:44.987822 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:44.987814    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:44.987945 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:44.987881    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:44.988021 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:44.988004    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:44.988154 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:44.988071    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:45.115691 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:45.115513    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27" event={"ID":"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163","Type":"ContainerStarted","Data":"b4c4955bb5b52c4fd63cb8af0e2b2cf020143185ef01c7ed2de1edf959c5209f"}
Nov 04 00:56:45.117848 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:45.117823    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wrcbh" event={"ID":"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb","Type":"ContainerStarted","Data":"9f24bd1cc1b97cca4291af8c541dce8920de7980234e03aae9ba5f8d5845fa59"}
Nov 04 00:56:45.120179 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:45.120155    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerStarted","Data":"5b9bc28ac2167a7e2e9b53b69a7ae0dacc4f0dad396ed4bd9563e14b0f406726"}
Nov 04 00:56:45.380695 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:45.380592    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27" podStartSLOduration=3.716740188 podStartE2EDuration="14.380577162s" totalImagesPullingTime="10.663836974s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:45.263833657 +0000 UTC m=+14.796094925" watchObservedRunningTime="2026-11-04 00:56:45.380577162 +0000 UTC m=+14.912838432"
Nov 04 00:56:45.494785 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:45.494733    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wrcbh" podStartSLOduration=3.639708923 podStartE2EDuration="14.494716042s" totalImagesPullingTime="10.855007119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:45.494547229 +0000 UTC m=+15.026808498" watchObservedRunningTime="2026-11-04 00:56:45.494716042 +0000 UTC m=+15.026977311"
Nov 04 00:56:46.124251 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:46.124209    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="5b9bc28ac2167a7e2e9b53b69a7ae0dacc4f0dad396ed4bd9563e14b0f406726" exitCode=0
Nov 04 00:56:46.124997 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:46.124260    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerDied","Data":"5b9bc28ac2167a7e2e9b53b69a7ae0dacc4f0dad396ed4bd9563e14b0f406726"}
Nov 04 00:56:46.987642 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:46.987613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:46.987811 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:46.987613    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:46.987811 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:46.987756    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:46.987811 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:46.987772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:46.987953 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:46.987866    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:46.987953 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:46.987941    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:48.131455 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.131211    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="23cee312cdb39e346c3274253fd4fb74962d4922feef44a0769dc16cfa2679b8" exitCode=0
Nov 04 00:56:48.131455 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.131302    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerDied","Data":"23cee312cdb39e346c3274253fd4fb74962d4922feef44a0769dc16cfa2679b8"}
Nov 04 00:56:48.686437 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.686287    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:48.686542 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.686488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:48.686542 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.686421    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:48.686542 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.686542    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:04.686528767 +0000 UTC m=+34.218790014 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 04 00:56:48.686727 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.686569    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:48.686727 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.686610    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret podName:d41414c4-11d2-4e45-aa39-8f33726a40fd nodeName:}" failed. No retries permitted until 2026-11-04 00:57:04.68660026 +0000 UTC m=+34.218861506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret") pod "global-pull-secret-syncer-rcwm8" (UID: "d41414c4-11d2-4e45-aa39-8f33726a40fd") : object "kube-system"/"original-pull-secret" not registered
Nov 04 00:56:48.787148 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.787118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:48.787264 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.787249    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 04 00:56:48.787300 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.787269    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 04 00:56:48.787300 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.787279    2581 projected.go:196] Error preparing data for projected volume kube-api-access-7l7cg for pod openshift-network-diagnostics/network-check-target-frjrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:48.787356 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.787318    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg podName:728c5532-b88d-4ccc-bb31-a0278f5f84f8 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:04.787305596 +0000 UTC m=+34.319566843 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l7cg" (UniqueName: "kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg") pod "network-check-target-frjrl" (UID: "728c5532-b88d-4ccc-bb31-a0278f5f84f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 04 00:56:48.987859 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.987825    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:48.987995 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.987842    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:48.987995 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.987964    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:48.987995 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:48.987992    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:48.988140 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.988043    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:48.988140 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:48.988109    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:49.136470 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:49.136403    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"67279a556a741319939ac4c92500777c514e81ca02482bccfd48994ba85344d0"}
Nov 04 00:56:49.136470 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:49.136465    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"1f7a86219f5b8004f465b7d2a2a1213a3d10eb806c3bef4e52f3528bf8dba3f0"}
Nov 04 00:56:49.136866 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:49.136494    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"1a73cbd8cdb559cae41787750aff41878f7de6245fb808f9a686abd7cbd58ae8"}
Nov 04 00:56:49.136866 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:49.136541    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"a298dd2a5fcc5ad44002f0a1a0b699d3b7954e2aecab8bcccb4845543ec20616"}
Nov 04 00:56:49.136866 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:49.136568    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"203e047aa67a2a4fd25da632e95d342360487b7404af2e5f06e40f88421d7d9d"}
Nov 04 00:56:49.136866 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:49.136594    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"4ab48eb931340f1ae2f5943a8588188518d9eb33acd40434c2b7233ed2786179"}
Nov 04 00:56:50.988603 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:50.988564    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:50.989094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:50.988690    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:50.989094 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:50.988706    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:50.989094 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:50.988729    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:50.989094 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:50.988795    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:50.989094 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:50.988842    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:51.143526 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:51.143438    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"b52791d8960ade25f96533cbaa90732c273b4462aa6db86e144acc5b602c63ad"}
Nov 04 00:56:51.146008 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:51.145984    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="fe701ee4c40ec837a2820badf8b2b8d3bcbfb0a014a315291ab7db28e8a49eda" exitCode=0
Nov 04 00:56:51.146008 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:51.146016    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerDied","Data":"fe701ee4c40ec837a2820badf8b2b8d3bcbfb0a014a315291ab7db28e8a49eda"}
Nov 04 00:56:52.150572 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:52.150541    2581 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="4548475b49e87977f600c40517a724d92559ad9a7f0cfee297ffc9404796a632" exitCode=0
Nov 04 00:56:52.151017 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:52.150584    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerDied","Data":"4548475b49e87977f600c40517a724d92559ad9a7f0cfee297ffc9404796a632"}
Nov 04 00:56:52.987971 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:52.987941    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:52.988126 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:52.987998    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:52.988126 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:52.988007    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:52.988211 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:52.988118    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:52.988261 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:52.988226    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:52.988318 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:52.988295    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:53.155585 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.155347    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" event={"ID":"85125fc3-99ec-427f-84a5-910d207e22cd","Type":"ContainerStarted","Data":"c659ef2d4ec750afbac163ee68261578fcdafdb736f5fb50d3c969c5d9f4d801"}
Nov 04 00:56:53.156162 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.155645    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:53.158792 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.158697    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" event={"ID":"b2fa4fe0-3667-44c4-963c-4572a330d29c","Type":"ContainerStarted","Data":"9ff9fd16d4e8ecf38cddccdb0fae4c498c8eb781678f8601f5a394cfcecb3e43"}
Nov 04 00:56:53.171308 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.171288    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:53.431918 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.431836    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" podStartSLOduration=7.202999999 podStartE2EDuration="22.431822126s" totalImagesPullingTime="15.228822127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:53.431375538 +0000 UTC m=+22.963636806" watchObservedRunningTime="2026-11-04 00:56:53.431822126 +0000 UTC m=+22.964083394"
Nov 04 00:56:53.590552 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.590515    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:53.671367 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.671307    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zb7k2" podStartSLOduration=9.060860834 podStartE2EDuration="22.671288991s" totalImagesPullingTime="13.610428157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:56:53.670177871 +0000 UTC m=+23.202439141" watchObservedRunningTime="2026-11-04 00:56:53.671288991 +0000 UTC m=+23.203550261"
Nov 04 00:56:53.792790 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.792700    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:53.973054 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:53.973017    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:54.164444 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.164360    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 00:56:54.165482 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.165461    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:54.181524 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.181494    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:54.190109 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.190078    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:54.792718 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.792686    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:54.957428 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.957391    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:54.987248 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.987218    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:54.987248 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.987238    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:54.987462 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:54.987315    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:54.987462 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:54.987322    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:54.987462 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:54.987384    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:54.987462 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:54.987439    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:55.166456 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.166389    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 00:56:55.459247 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.459216    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:56:55.625326 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.625289    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frjrl"]
Nov 04 00:56:55.625489 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.625409    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:55.625547 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:55.625525    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:55.628154 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.628130    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rcwm8"]
Nov 04 00:56:55.628268 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.628204    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:55.628319 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:55.628281    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:55.628725 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.628705    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7vlfq"]
Nov 04 00:56:55.628807 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:55.628799    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:55.628909 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:55.628891    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:57.181782 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:57.181740    2581 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-w557m" podUID="85125fc3-99ec-427f-84a5-910d207e22cd" containerName="ovnkube-controller" probeResult="failure" output=""
Nov 04 00:56:57.988138 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:57.988071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:57.988138 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:57.988090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:57.988300 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:57.988080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:57.988300 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:57.988163    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:57.988300 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:57.988235    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:57.988393 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:57.988317    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:56:58.277365 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:58.277283    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:56:59.987961 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:59.987886    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:56:59.988420 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:59.987991    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:56:59.988420 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:59.988003    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rcwm8" podUID="d41414c4-11d2-4e45-aa39-8f33726a40fd"
Nov 04 00:56:59.988420 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:56:59.988016    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:56:59.988420 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:59.988079    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-frjrl" podUID="728c5532-b88d-4ccc-bb31-a0278f5f84f8"
Nov 04 00:56:59.988420 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:56:59.988148    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7vlfq" podUID="c5dca767-45ed-4760-9d96-d2db34951608"
Nov 04 00:57:00.755557 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.755383    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeReady"
Nov 04 00:57:00.755722 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.755645    2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 04 00:57:00.867753 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.867681    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5hk4x"]
Nov 04 00:57:00.913394 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.913360    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qmqjk"]
Nov 04 00:57:00.913550 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.913477    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:00.931579 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.931558    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:00.932823 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.932794    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qmqjk"]
Nov 04 00:57:00.935565 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.935544    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5hk4x"]
Nov 04 00:57:00.966217 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.966188    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 00:57:00.966353 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.966220    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 00:57:00.966353 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.966331    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8chh7\""
Nov 04 00:57:00.995777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.995753    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 00:57:00.995777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:00.995775    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-g2z88\""
Nov 04 00:57:01.022042 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.022014    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:01.066349 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.066320    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 00:57:01.066349 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.066350    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 00:57:01.075591 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.075567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/590feae8-76fd-4069-95b5-fd8b213b5471-tmp-dir\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.075708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.075596    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/590feae8-76fd-4069-95b5-fd8b213b5471-config-volume\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.075708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.075621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp6xp\" (UniqueName: \"kubernetes.io/projected/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-kube-api-access-lp6xp\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.075708 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.075638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.075847 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.075827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x64x6\" (UniqueName: \"kubernetes.io/projected/590feae8-76fd-4069-95b5-fd8b213b5471-kube-api-access-x64x6\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.075883 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.075870    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-cert\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.176930 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.176849    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x64x6\" (UniqueName: \"kubernetes.io/projected/590feae8-76fd-4069-95b5-fd8b213b5471-kube-api-access-x64x6\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.176930 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.176896    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-cert\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.177144 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.177020    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/590feae8-76fd-4069-95b5-fd8b213b5471-tmp-dir\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.177144 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.177060    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/590feae8-76fd-4069-95b5-fd8b213b5471-config-volume\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.177144 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.177090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6xp\" (UniqueName: \"kubernetes.io/projected/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-kube-api-access-lp6xp\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.177144 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.177118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.177340 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:01.177224    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:01.177340 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:01.177285    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:01.67726734 +0000 UTC m=+31.209528589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:01.177453 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.177371    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/590feae8-76fd-4069-95b5-fd8b213b5471-tmp-dir\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.177716 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.177693    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/590feae8-76fd-4069-95b5-fd8b213b5471-config-volume\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.181178 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.181155    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-cert\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.189124 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.189090    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:01.229010 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.228989    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"]
Nov 04 00:57:01.236482 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.236458    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x64x6\" (UniqueName: \"kubernetes.io/projected/590feae8-76fd-4069-95b5-fd8b213b5471-kube-api-access-x64x6\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.238565 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.238544    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6xp\" (UniqueName: \"kubernetes.io/projected/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-kube-api-access-lp6xp\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.253177 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.253160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:01.264008 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.263989    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:01.286413 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.286392    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"]
Nov 04 00:57:01.308366 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.308336    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 00:57:01.308632 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.308608    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 00:57:01.308738 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.308619    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-p4bs4\""
Nov 04 00:57:01.372845 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.372797    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:01.378702 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.378684    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjx6z\" (UniqueName: \"kubernetes.io/projected/d34f4e73-e682-4ad2-acdf-ff6209283ace-kube-api-access-kjx6z\") pod \"migrator-66df5d496b-mm5vx\" (UID: \"d34f4e73-e682-4ad2-acdf-ff6209283ace\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:01.479204 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.479130    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjx6z\" (UniqueName: \"kubernetes.io/projected/d34f4e73-e682-4ad2-acdf-ff6209283ace-kube-api-access-kjx6z\") pod \"migrator-66df5d496b-mm5vx\" (UID: \"d34f4e73-e682-4ad2-acdf-ff6209283ace\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:01.542387 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.542358    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjx6z\" (UniqueName: \"kubernetes.io/projected/d34f4e73-e682-4ad2-acdf-ff6209283ace-kube-api-access-kjx6z\") pod \"migrator-66df5d496b-mm5vx\" (UID: \"d34f4e73-e682-4ad2-acdf-ff6209283ace\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:01.543106 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.543058    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:01.567126 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.567097    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:01.658234 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.658205    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qmqjk"]
Nov 04 00:57:01.663050 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:57:01.663022    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3ecba1b_5ec4_41f0_b49a_9d8167d31114.slice/crio-2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2 WatchSource:0}: Error finding container 2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2: Status 404 returned error can't find the container with id 2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2
Nov 04 00:57:01.680842 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.680816    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:01.680953 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:01.680924    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:01.680999 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:01.680989    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:02.680970364 +0000 UTC m=+32.213231612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:01.710854 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.710821    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:01.966754 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.966681    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"]
Nov 04 00:57:01.971590 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:57:01.971565    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd34f4e73_e682_4ad2_acdf_ff6209283ace.slice/crio-0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00 WatchSource:0}: Error finding container 0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00: Status 404 returned error can't find the container with id 0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00
Nov 04 00:57:01.987457 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.987427    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:01.987558 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.987486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:01.987558 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:01.987514    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:02.040966 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.040910    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 00:57:02.040966 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.040918    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-47hrz\""
Nov 04 00:57:02.040966 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.040918    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 00:57:02.041777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.040980    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fh9wk\""
Nov 04 00:57:02.041777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.041126    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-xq5nc\""
Nov 04 00:57:02.041777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.041138    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 00:57:02.041777 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.041219    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 00:57:02.182382 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.182345    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx" event={"ID":"d34f4e73-e682-4ad2-acdf-ff6209283ace","Type":"ContainerStarted","Data":"0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00"}
Nov 04 00:57:02.187404 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.187374    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qmqjk" event={"ID":"a3ecba1b-5ec4-41f0-b49a-9d8167d31114","Type":"ContainerStarted","Data":"2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2"}
Nov 04 00:57:02.260878 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.260803    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-26d2d"]
Nov 04 00:57:02.281154 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.281124    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.322407 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.322374    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-26d2d"]
Nov 04 00:57:02.385598 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.385556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.385598 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.385601    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vls2r\" (UniqueName: \"kubernetes.io/projected/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-api-access-vls2r\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.385862 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.385713    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.385862 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.385773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-crio-socket\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.385862 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.385810    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-data-volume\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.421989 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.421958    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 00:57:02.422191 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.422175    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 00:57:02.422250 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.422175    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 00:57:02.422250 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.422178    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 00:57:02.422321 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.422249    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b7knl\""
Nov 04 00:57:02.486755 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.486713    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-crio-socket\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.486944 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.486766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-data-volume\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.486944 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.486806    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.486944 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.486828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vls2r\" (UniqueName: \"kubernetes.io/projected/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-api-access-vls2r\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.486944 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.486926    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-crio-socket\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.486944 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.486932    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 00:57:02.487183 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.486999    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.487183 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.487012    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:02.986993006 +0000 UTC m=+32.519254278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : configmap "kube-rbac-proxy" not found
Nov 04 00:57:02.487183 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.487140    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-data-volume\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.487183 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.487148    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:02.487352 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.487222    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:02.987201452 +0000 UTC m=+32.519462711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:02.489253 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.489227    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:02.547226 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.547157    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vls2r\" (UniqueName: \"kubernetes.io/projected/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-api-access-vls2r\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.640323 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.640286    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:02.688800 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.688752    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:02.688968 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.688901    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:02.688968 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.688964    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:04.688947849 +0000 UTC m=+34.221209109 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:02.933594 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.933444    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:02.991640 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.991583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.991884 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:02.991714    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:02.991884 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.991836    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 00:57:02.992012 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.991903    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:03.991885588 +0000 UTC m=+33.524146838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : configmap "kube-rbac-proxy" not found
Nov 04 00:57:02.992090 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.992066    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:02.992144 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:02.992135    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:03.992102912 +0000 UTC m=+33.524364179 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:03.091523 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:03.091488    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:03.261641 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:03.261539    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:03.417297 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:03.417258    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:04.000562 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.000528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:04.000900 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.000598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:04.000900 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.000712    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 04 00:57:04.000900 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.000785    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:04.000900 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.000804    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:06.000782999 +0000 UTC m=+35.533044262 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : configmap "kube-rbac-proxy" not found
Nov 04 00:57:04.000900 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.000843    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:06.000825615 +0000 UTC m=+35.533086879 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:04.417820 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.417750    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:04.600773 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.600659    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:04.707574 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.707488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:04.707574 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.707549    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:04.707834 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.707585    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:04.707834 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.707658    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:04.707834 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.707693    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 04 00:57:04.707834 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.707755    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:08.707736446 +0000 UTC m=+38.239997693 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:04.707834 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:04.707775    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs podName:c5dca767-45ed-4760-9d96-d2db34951608 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:36.707765914 +0000 UTC m=+66.240027164 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs") pod "network-metrics-daemon-7vlfq" (UID: "c5dca767-45ed-4760-9d96-d2db34951608") : secret "metrics-daemon-secret" not found
Nov 04 00:57:04.710315 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.710295    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:04.758743 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.758707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:04.808143 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.808113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:04.811226 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.811204    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:04.996320 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:04.996285    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:05.054487 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.054457    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:05.169610 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.169582    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rcwm8"]
Nov 04 00:57:05.175055 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:57:05.175027    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41414c4_11d2_4e45_aa39_8f33726a40fd.slice/crio-430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5 WatchSource:0}: Error finding container 430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5: Status 404 returned error can't find the container with id 430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5
Nov 04 00:57:05.197419 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.197382    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx" event={"ID":"d34f4e73-e682-4ad2-acdf-ff6209283ace","Type":"ContainerStarted","Data":"5cf9710839c82cda5e74b3309522447d88c408f3b6cd90ba89fc451e4975adfc"}
Nov 04 00:57:05.198684 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.198637    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rcwm8" event={"ID":"d41414c4-11d2-4e45-aa39-8f33726a40fd","Type":"ContainerStarted","Data":"430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5"}
Nov 04 00:57:05.240958 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.240931    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:05.313563 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.313505    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx" podStartSLOduration=1.8610820270000001 podStartE2EDuration="4.313487375s" totalImagesPullingTime="2.452405348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:05.312610067 +0000 UTC m=+34.844871336" watchObservedRunningTime="2026-11-04 00:57:05.313487375 +0000 UTC m=+34.845748646"
Nov 04 00:57:05.453598 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:05.453569    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-frjrl"]
Nov 04 00:57:05.457363 ip-10-0-136-128 kubenswrapper[2581]: W1104 00:57:05.457338    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod728c5532_b88d_4ccc_bb31_a0278f5f84f8.slice/crio-0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7 WatchSource:0}: Error finding container 0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7: Status 404 returned error can't find the container with id 0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7
Nov 04 00:57:06.021714 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:06.021620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:06.021714 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:06.021706    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:06.021928 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:06.021795    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:06.021928 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:06.021887    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:10.02185438 +0000 UTC m=+39.554115628 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:06.022224 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:06.022204    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:06.202443 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:06.202405    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qmqjk" event={"ID":"a3ecba1b-5ec4-41f0-b49a-9d8167d31114","Type":"ContainerStarted","Data":"9e4b53078a9ffc30640c67d28fb30bd1e234a0425968e7054b06a4ad21ef7c53"}
Nov 04 00:57:06.206879 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:06.206835    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frjrl" event={"ID":"728c5532-b88d-4ccc-bb31-a0278f5f84f8","Type":"ContainerStarted","Data":"0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7"}
Nov 04 00:57:06.316573 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:06.316484    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qmqjk" podStartSLOduration=2.252685236 podStartE2EDuration="6.316470664s" totalImagesPullingTime="4.063785428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:06.315903085 +0000 UTC m=+35.848164357" watchObservedRunningTime="2026-11-04 00:57:06.316470664 +0000 UTC m=+35.848731966"
Nov 04 00:57:08.743949 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:08.743913    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:08.744429 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:08.744116    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:08.744429 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:08.744183    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:16.744164892 +0000 UTC m=+46.276426153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:08.788452 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:08.788354    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:09.002369 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:09.002284    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:10.054547 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.054442    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:10.054951 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:10.054614    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:10.054951 ip-10-0-136-128 kubenswrapper[2581]: E1104 00:57:10.054707    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls podName:251c150b-e0d4-455b-8cdd-6a88e2fa0e48 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:18.054686069 +0000 UTC m=+47.586947318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls") pod "insights-runtime-extractor-26d2d" (UID: "251c150b-e0d4-455b-8cdd-6a88e2fa0e48") : secret "insights-runtime-extractor-tls" not found
Nov 04 00:57:10.074565 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.074537    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rc99q_352db7a3-daed-4f66-81db-7054c082d043/dns-node-resolver/0.log"
Nov 04 00:57:10.220343 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.220307    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-frjrl" event={"ID":"728c5532-b88d-4ccc-bb31-a0278f5f84f8","Type":"ContainerStarted","Data":"c450d8c3dbab64eb0c6c7cc8d09c1788b8fc5d898105fdddde2cd0384d6f48ff"}
Nov 04 00:57:10.220600 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.220578    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:10.332842 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.332743    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-frjrl" podStartSLOduration=35.05776015 podStartE2EDuration="39.332727341s" totalImagesPullingTime="4.274967191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:10.332066033 +0000 UTC m=+39.864327301" watchObservedRunningTime="2026-11-04 00:57:10.332727341 +0000 UTC m=+39.864988612"
Nov 04 00:57:10.720135 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.720098    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:10.878904 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.878879    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6sgf4_5f018fcc-7d38-494c-bfe5-67dc725ec5a7/node-ca/0.log"
Nov 04 00:57:10.985541 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:10.985517    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:11.144715 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:11.144672    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:11.224063 ip-10-0-136-128 kubenswrapper[2581]: I1104 00:57:11.224026    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rcwm8" event={"ID":"d41414c4-11d2-4e45-aa39-8f33726a40fd","Type":"ContainerStarted","Data":"9d06c68f0ec03b42c95e4614d13368fd87bd66ee9fa31a15e20345307548a7c7"}
Nov 04 00:57:11.272127 ip-10-0-136-128 systemd[1]: Stopping Kubernetes Kubelet...
Nov 04 00:57:11.293209 ip-10-0-136-128 systemd[1]: kubelet.service: Deactivated successfully.
Nov 04 00:57:11.293465 ip-10-0-136-128 systemd[1]: Stopped Kubernetes Kubelet.
Nov 04 00:57:11.293749 ip-10-0-136-128 systemd[1]: kubelet.service: Consumed 2.312s CPU time.
Nov 04 00:57:11.309077 ip-10-0-136-128 systemd[1]: Starting Kubernetes Kubelet...
Nov 04 00:57:11.391917 ip-10-0-136-128 kubenswrapper[4434]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 00:57:11.391917 ip-10-0-136-128 kubenswrapper[4434]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 04 00:57:11.391917 ip-10-0-136-128 kubenswrapper[4434]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 00:57:11.391917 ip-10-0-136-128 kubenswrapper[4434]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 04 00:57:11.393914 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393840    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.393914 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393913    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.393914 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393917    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393920    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393924    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393927    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393930    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393933    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393935    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393938    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393942    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393945    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393948    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393951    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393953    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393956    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393959    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393962    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393964    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393968    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393970    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.394010 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393973    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393976    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393978    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393981    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393985    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393987    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393992    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393996    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.393999    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394002    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394005    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394007    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394010    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394013    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394015    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394018    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394021    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394024    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394026    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394029    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.394489 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394031    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394034    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394037    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394040    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394042    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394045    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394048    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394050    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394053    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394057    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394059    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394062    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394065    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394068    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394071    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394074    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394077    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394080    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394082    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.395007 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394085    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394089    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394093    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394096    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394100    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394103    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394106    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394108    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394111    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394114    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394117    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394120    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394123    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394125    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394128    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394130    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394133    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394135    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394138    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.395474 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394140    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394143    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394146    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394148    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394151    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394154    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394157    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394159    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394162    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394165    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394168    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394170    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394173    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394175    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394178    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394180    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394183    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394185    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394188    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394190    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.395961 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394192    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394195    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394198    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394200    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394204    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394208    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.394210    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395099    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395108    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395111    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395115    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395117    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395120    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395123    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395126    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395128    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395132    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395135    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395138    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395142    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.396484 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395146    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395149    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395152    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395155    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395157    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395160    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395163    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395165    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395168    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395170    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395173    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395175    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395178    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395181    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395183    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395186    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395188    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395191    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395195    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395197    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.397003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395200    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395203    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395206    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395208    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395211    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395215    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395219    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395222    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395224    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395227    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395230    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395232    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395235    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395239    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395241    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395244    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395247    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395250    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395253    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395255    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.397509 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395258    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395261    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395263    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395266    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395269    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395271    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395274    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395277    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395280    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395283    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395286    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395295    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395298    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395300    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395303    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395305    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395308    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395310    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395313    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.398012 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395316    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395318    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395321    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395323    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395326    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395328    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395331    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395334    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395336    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395339    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395342    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395344    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395346    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395349    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395352    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395354    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395356    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395359    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395361    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.398540 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395364    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395367    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395369    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395372    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395375    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395377    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395387    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395390    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395392    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395396    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395398    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395401    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395404    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395406    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395409    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395911    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395917    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395920    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395924    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.399026 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395927    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395929    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395932    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395935    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395937    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395940    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395943    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395945    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395948    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395951    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395953    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395959    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395962    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395964    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395968    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395970    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395973    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395976    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395979    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395984    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.399503 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395987    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395990    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395992    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395996    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.395999    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396001    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396004    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396007    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396011    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396015    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396018    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396020    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396023    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396026    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396029    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396031    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396034    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396036    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396039    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396041    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.400253 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396044    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396046    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396049    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396054    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396063    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396066    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396069    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396072    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396075    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396077    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396080    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396083    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396086    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396089    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396092    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396096    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396098    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396101    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396103    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396106    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.400963 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396108    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396111    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396114    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396116    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396119    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396122    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396124    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396127    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396130    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396134    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396138    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396141    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396144    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396147    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396150    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396154    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396157    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396159    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396162    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.401461 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396165    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396167    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396170    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396173    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396175    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396178    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396181    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396183    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396186    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396189    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396192    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396194    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396197    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396199    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396202    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396204    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396207    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396210    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396213    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.401945 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396215    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396217    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396221    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396223    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396316    4434 flags.go:64] FLAG: --address="0.0.0.0"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396324    4434 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396331    4434 flags.go:64] FLAG: --anonymous-auth="true"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396335    4434 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396341    4434 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396352    4434 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396358    4434 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396365    4434 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396368    4434 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396372    4434 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396375    4434 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396379    4434 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396382    4434 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396385    4434 flags.go:64] FLAG: --cgroup-root=""
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396388    4434 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396391    4434 flags.go:64] FLAG: --client-ca-file=""
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396395    4434 flags.go:64] FLAG: --cloud-provider="external"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396398    4434 flags.go:64] FLAG: --cluster-dns="[]"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396402    4434 flags.go:64] FLAG: --cluster-domain=""
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396405    4434 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 04 00:57:11.402421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396408    4434 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396411    4434 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396415    4434 flags.go:64] FLAG: --container-log-max-files="5"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396419    4434 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396422    4434 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396425    4434 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396428    4434 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396431    4434 flags.go:64] FLAG: --contention-profiling="false"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396434    4434 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396437    4434 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396440    4434 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396445    4434 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396449    4434 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396453    4434 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396455    4434 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396458    4434 flags.go:64] FLAG: --enable-load-reader="false"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396461    4434 flags.go:64] FLAG: --enable-server="true"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396466    4434 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396470    4434 flags.go:64] FLAG: --event-burst="100"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396474    4434 flags.go:64] FLAG: --event-qps="50"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396477    4434 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396480    4434 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396483    4434 flags.go:64] FLAG: --eviction-hard=""
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396487    4434 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 04 00:57:11.403023 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396490    4434 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396493    4434 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396497    4434 flags.go:64] FLAG: --eviction-soft=""
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396500    4434 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396503    4434 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396506    4434 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396509    4434 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396512    4434 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396515    4434 flags.go:64] FLAG: --fail-swap-on="true"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396518    4434 flags.go:64] FLAG: --feature-gates=""
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396522    4434 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396525    4434 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396528    4434 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396532    4434 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396535    4434 flags.go:64] FLAG: --healthz-port="10248"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396538    4434 flags.go:64] FLAG: --help="false"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396541    4434 flags.go:64] FLAG: --hostname-override="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396544    4434 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396547    4434 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396554    4434 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396558    4434 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396561    4434 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396564    4434 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 04 00:57:11.403650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396567    4434 flags.go:64] FLAG: --image-service-endpoint=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396570    4434 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396574    4434 flags.go:64] FLAG: --kube-api-burst="100"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396577    4434 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396580    4434 flags.go:64] FLAG: --kube-api-qps="50"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396583    4434 flags.go:64] FLAG: --kube-reserved=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396586    4434 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396589    4434 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396592    4434 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396595    4434 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396598    4434 flags.go:64] FLAG: --lock-file=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396601    4434 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396604    4434 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396607    4434 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396612    4434 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396616    4434 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396619    4434 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396622    4434 flags.go:64] FLAG: --logging-format="text"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396625    4434 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396628    4434 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396631    4434 flags.go:64] FLAG: --manifest-url=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396634    4434 flags.go:64] FLAG: --manifest-url-header=""
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396638    4434 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396641    4434 flags.go:64] FLAG: --max-open-files="1000000"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396645    4434 flags.go:64] FLAG: --max-pods="110"
Nov 04 00:57:11.404224 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396649    4434 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396652    4434 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396655    4434 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396659    4434 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396673    4434 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396676    4434 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396679    4434 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396686    4434 flags.go:64] FLAG: --node-status-max-images="50"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396689    4434 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396694    4434 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396697    4434 flags.go:64] FLAG: --pod-cidr=""
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396700    4434 flags.go:64] FLAG: --pod-manifest-path=""
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396703    4434 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396707    4434 flags.go:64] FLAG: --pods-per-core="0"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396710    4434 flags.go:64] FLAG: --port="10250"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396713    4434 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396715    4434 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0e6c45a8a47a38687"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396719    4434 flags.go:64] FLAG: --qos-reserved=""
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396722    4434 flags.go:64] FLAG: --read-only-port="10255"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396725    4434 flags.go:64] FLAG: --register-node="true"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396728    4434 flags.go:64] FLAG: --register-with-taints=""
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396732    4434 flags.go:64] FLAG: --registry-burst="10"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396735    4434 flags.go:64] FLAG: --registry-qps="5"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396738    4434 flags.go:64] FLAG: --reserved-cpus=""
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396741    4434 flags.go:64] FLAG: --reserved-memory=""
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396745    4434 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 04 00:57:11.405075 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396748    4434 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396751    4434 flags.go:64] FLAG: --rotate-certificates="false"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396754    4434 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396757    4434 flags.go:64] FLAG: --runonce="false"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396760    4434 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396763    4434 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396766    4434 flags.go:64] FLAG: --seccomp-default="false"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396769    4434 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396772    4434 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396776    4434 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396779    4434 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396783    4434 flags.go:64] FLAG: --storage-driver-password="root"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396785    4434 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396788    4434 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396791    4434 flags.go:64] FLAG: --storage-driver-user="root"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396796    4434 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396799    4434 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396802    4434 flags.go:64] FLAG: --system-cgroups=""
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396805    4434 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396811    4434 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396814    4434 flags.go:64] FLAG: --tls-cert-file=""
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396816    4434 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396820    4434 flags.go:64] FLAG: --tls-min-version=""
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396823    4434 flags.go:64] FLAG: --tls-private-key-file=""
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396826    4434 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 04 00:57:11.405737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396829    4434 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396832    4434 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396835    4434 flags.go:64] FLAG: --v="2"
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396839    4434 flags.go:64] FLAG: --version="false"
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396843    4434 flags.go:64] FLAG: --vmodule=""
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396847    4434 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.396851    4434 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396976    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396980    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396982    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396985    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396988    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396990    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396993    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396996    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.396998    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397002    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397005    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397008    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397011    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397014    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.406555 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397017    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397020    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397023    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397026    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397028    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397031    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397033    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397036    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397039    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397041    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397044    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397046    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397049    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397052    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397054    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397057    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397060    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397063    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397066    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.407235 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397069    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397071    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397074    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397077    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397080    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397082    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397085    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397087    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397091    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397094    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397097    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397099    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397102    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397104    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397107    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397110    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397112    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397115    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397118    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.407779 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397120    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397123    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397125    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397128    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397130    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397133    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397136    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397138    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397141    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397143    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397146    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397149    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397152    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397155    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397157    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397160    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397162    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397165    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397168    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397170    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.408394 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397173    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397176    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397179    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397182    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397185    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397187    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397191    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397195    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397198    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397201    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397203    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397206    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397208    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397211    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397214    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397216    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397219    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397221    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397224    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.408915 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397226    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397229    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397231    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397234    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397237    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397239    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397242    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397245    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397247    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397250    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397253    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397255    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397258    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397260    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397265    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.397294    4434 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 00:57:11.409534 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397327    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397331    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397334    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397337    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397340    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397344    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397347    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397350    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397353    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397356    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397358    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397361    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397363    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397366    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397368    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397371    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397374    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397376    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397379    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.410008 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397381    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397386    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397389    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397391    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397395    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397397    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397400    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397403    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397406    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397408    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397411    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397415    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397418    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397420    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397422    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397425    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397428    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397430    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397433    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397435    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.410513 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397438    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397441    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397443    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397446    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397450    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397453    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397456    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397459    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397461    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397464    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397466    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397469    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397472    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397475    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397478    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397481    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397483    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397486    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397489    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397492    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.411131 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397495    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397497    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397500    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397504    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397507    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397509    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397512    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397515    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397518    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397520    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397523    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397527    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397531    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397534    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397537    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397540    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397543    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397545    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397548    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.411693 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397550    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397553    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397555    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397558    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397561    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397563    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397568    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397570    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397573    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397575    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397578    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397581    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397584    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397586    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397589    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397592    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397595    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397598    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397601    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397603    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.412166 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397606    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397609    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397612    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397614    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397617    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397619    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397623    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.397625    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.402061    4434 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.402079    4434 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.402100    4434 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.402105    4434 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402180    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402185    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402188    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402191    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402194    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402197    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402199    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.412801 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402202    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402205    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402207    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402210    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402213    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402215    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402218    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402221    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402224    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402227    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402229    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402232    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402234    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402237    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402240    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402243    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402246    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402249    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402251    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.413455 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402254    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402257    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402260    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402263    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402266    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402269    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402271    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402274    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402277    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402279    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402282    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402285    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402287    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402290    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402293    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402295    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402298    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402300    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402303    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402306    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.414033 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402309    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402311    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402314    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402316    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402320    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402322    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402328    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402332    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402335    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402338    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402341    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402344    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402347    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402349    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402352    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402355    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402358    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402361    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402364    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402367    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.414715 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402369    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402372    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402374    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402377    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402380    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402382    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402385    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402388    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402390    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402393    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402396    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402398    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402401    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402403    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402406    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402408    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402412    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402414    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402418    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.415342 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402423    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402426    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402429    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402432    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402435    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402437    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402440    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402443    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402446    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402448    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402451    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402453    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402456    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402459    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402462    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402464    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402467    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402470    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402472    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.416227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402475    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402477    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.402504    4434 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402640    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402645    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402648    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402651    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402653    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402657    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402660    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402677    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402681    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402684    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402686    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402689    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402692    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.417003 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402694    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402697    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402699    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402702    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402705    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402707    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402710    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402712    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402715    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402717    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402720    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402723    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402726    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402729    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402731    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402734    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402736    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402739    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402741    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402744    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.417577 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402746    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402749    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402751    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402754    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402756    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402760    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402763    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402766    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402769    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402772    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402774    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402777    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402781    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402785    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402788    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402790    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402793    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402796    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402799    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.418130 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402802    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402804    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402807    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402810    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402812    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402815    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402818    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402820    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402823    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402826    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402828    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402831    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402833    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402836    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402838    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402841    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402843    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402846    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402849    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.418884 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402852    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402855    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402857    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402860    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402863    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402865    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402868    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402870    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402873    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402875    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402878    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402880    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402883    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402885    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402888    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402890    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402893    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402895    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402899    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.419647 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402903    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402906    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402909    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402911    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402914    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402916    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402919    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402922    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402924    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402927    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402929    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402932    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402934    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402937    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402941    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402944    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.420454 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.402968    4434 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.402999    4434 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403003    4434 feature_gate.go:386] unrecognized feature gate: Example2
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403006    4434 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403009    4434 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403013    4434 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403016    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403019    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403022    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403025    4434 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403029    4434 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403032    4434 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403034    4434 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403038    4434 feature_gate.go:386] unrecognized feature gate: Example
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403041    4434 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403044    4434 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403046    4434 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403050    4434 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403054    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 04 00:57:11.420940 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403057    4434 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403060    4434 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403063    4434 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403065    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403068    4434 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403070    4434 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403073    4434 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403078    4434 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403081    4434 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403084    4434 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403086    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403089    4434 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403093    4434 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403095    4434 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403098    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403101    4434 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403103    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403106    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403108    4434 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403111    4434 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 04 00:57:11.421413 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403114    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403116    4434 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403119    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403121    4434 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403124    4434 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403127    4434 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403129    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403132    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403134    4434 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403137    4434 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403139    4434 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403142    4434 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403145    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403147    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403150    4434 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403152    4434 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403155    4434 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403157    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403160    4434 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 04 00:57:11.421922 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403166    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403169    4434 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403172    4434 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403176    4434 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403181    4434 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403184    4434 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403187    4434 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403190    4434 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403193    4434 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403196    4434 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403198    4434 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403201    4434 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403204    4434 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403206    4434 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403209    4434 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403211    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403214    4434 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403217    4434 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403219    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 04 00:57:11.422399 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403222    4434 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403225    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403227    4434 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403230    4434 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403233    4434 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403235    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403238    4434 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403241    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403243    4434 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403246    4434 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403248    4434 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403251    4434 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403254    4434 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403258    4434 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403260    4434 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403263    4434 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403266    4434 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403270    4434 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403272    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403275    4434 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 04 00:57:11.422909 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403278    4434 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403281    4434 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403284    4434 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403286    4434 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403289    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403292    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403295    4434 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403298    4434 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403301    4434 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:11.403304    4434 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.403356    4434 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404310    4434 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404360    4434 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404528    4434 server.go:1014] "Starting client certificate rotation"
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404698    4434 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404769    4434 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 00:51:31 +0000 UTC" deadline="2028-06-02 00:54:16.426047938 +0000 UTC"
Nov 04 00:57:11.423396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404809    4434 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13823h57m5.021242543s"
Nov 04 00:57:11.423800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.404968    4434 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 00:57:11.423800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.406066    4434 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 04 00:57:11.423800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.407550    4434 log.go:25] "Validated CRI v1 runtime API"
Nov 04 00:57:11.423800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.409071    4434 log.go:25] "Validated CRI v1 image API"
Nov 04 00:57:11.423800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.409881    4434 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 04 00:57:11.423800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.411230    4434 fs.go:135] Filesystem UUIDs: map[01087df5-4860-4dad-912f-19ddf08b71a3:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 88f2c2f9-d576-47f4-b068-c02e9645b516:/dev/nvme0n1p4]
Nov 04 00:57:11.423966 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.411249    4434 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48/userdata/shm major:0 minor:73 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/251c150b-e0d4-455b-8cdd-6a88e2fa0e48/volumes/kubernetes.io~projected/kube-api-access-vls2r:{mountpoint:/var/lib/kubelet/pods/251c150b-e0d4-455b-8cdd-6a88e2fa0e48/volumes/kubernetes.io~projected/kube-api-access-vls2r major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/352db7a3-daed-4f66-81db-7054c082d043/volumes/kubernetes.io~projected/kube-api-access-bf2sc:{mountpoint:/var/lib/kubelet/pods/352db7a3-daed-4f66-81db-7054c082d043/volumes/kubernetes.io~projected/kube-api-access-bf2sc major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4813a1fc-7a03-487f-b9fa-8d1a7250768f/volumes/kubernetes.io~projected/kube-api-access-wrp79:{mountpoint:/var/lib/kubelet/pods/4813a1fc-7a03-487f-b9fa-8d1a7250768f/volumes/kubernetes.io~projected/kube-api-access-wrp79 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/590feae8-76fd-4069-95b5-fd8b213b5471/volumes/kubernetes.io~projected/kube-api-access-x64x6:{mountpoint:/var/lib/kubelet/pods/590feae8-76fd-4069-95b5-fd8b213b5471/volumes/kubernetes.io~projected/kube-api-access-x64x6 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f018fcc-7d38-494c-bfe5-67dc725ec5a7/volumes/kubernetes.io~projected/kube-api-access-qt6xz:{mountpoint:/var/lib/kubelet/pods/5f018fcc-7d38-494c-bfe5-67dc725ec5a7/volumes/kubernetes.io~projected/kube-api-access-qt6xz major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~empty-dir/tmp major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~projected/kube-api-access-25ppk:{mountpoint:/var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~projected/kube-api-access-25ppk major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/728c5532-b88d-4ccc-bb31-a0278f5f84f8/volumes/kubernetes.io~projected/kube-api-access-7l7cg:{mountpoint:/var/lib/kubelet/pods/728c5532-b88d-4ccc-bb31-a0278f5f84f8/volumes/kubernetes.io~projected/kube-api-access-7l7cg major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volumes/kubernetes.io~projected/kube-api-access-gggw2:{mountpoint:/var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volumes/kubernetes.io~projected/kube-api-access-gggw2 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3ecba1b-5ec4-41f0-b49a-9d8167d31114/volumes/kubernetes.io~projected/kube-api-access-lp6xp:{mountpoint:/var/lib/kubelet/pods/a3ecba1b-5ec4-41f0-b49a-9d8167d31114/volumes/kubernetes.io~projected/kube-api-access-lp6xp major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3ecba1b-5ec4-41f0-b49a-9d8167d31114/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/a3ecba1b-5ec4-41f0-b49a-9d8167d31114/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b2fa4fe0-3667-44c4-963c-4572a330d29c/volumes/kubernetes.io~projected/kube-api-access-wm766:{mountpoint:/var/lib/kubelet/pods/b2fa4fe0-3667-44c4-963c-4572a330d29c/volumes/kubernetes.io~projected/kube-api-access-wm766 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c5dca767-45ed-4760-9d96-d2db34951608/volumes/kubernetes.io~projected/kube-api-access-6qvnz:{mountpoint:/var/lib/kubelet/pods/c5dca767-45ed-4760-9d96-d2db34951608/volumes/kubernetes.io~projected/kube-api-access-6qvnz major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d34f4e73-e682-4ad2-acdf-ff6209283ace/volumes/kubernetes.io~projected/kube-api-access-kjx6z:{mountpoint:/var/lib/kubelet/pods/d34f4e73-e682-4ad2-acdf-ff6209283ace/volumes/kubernetes.io~projected/kube-api-access-kjx6z major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d41414c4-11d2-4e45-aa39-8f33726a40fd/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/d41414c4-11d2-4e45-aa39-8f33726a40fd/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de508010-fa70-4f37-a691-bdecaecdd3e7/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/de508010-fa70-4f37-a691-bdecaecdd3e7/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163/volumes/kubernetes.io~projected/kube-api-access-pmbpl:{mountpoint:/var/lib/kubelet/pods/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163/volumes/kubernetes.io~projected/kube-api-access-pmbpl major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef3c5a66-a065-45ac-8979-a13b2e382041/volumes/kubernetes.io~projected/kube-api-access-5dbxh:{mountpoint:/var/lib/kubelet/pods/ef3c5a66-a065-45ac-8979-a13b2e382041/volumes/kubernetes.io~projected/kube-api-access-5dbxh major:0 minor:59 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/3fa9d451003c82c5b243f3fe1fbc458d5e75fa561c81ecf6c5d90d319748fe0d/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c2ccd966fb1d06d30611608e5b9bc9155f650a7d3b69b8c3e75d9a30d99ae7a9/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/f7849daac376939b0aee3261925c7ca744c2a47832a3ce97866cd83d99a5ed43/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/3b550e6ddcd07713cd31ca6492f0e46c170c84cce60f42c07f5aea0e2b92f965/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/76d532befd2e0a4793f529f2f02cf83f7e8efd479476fba61b417b45c621395f/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/84121724d920041ab1949b02bb1d39be95ab9b206546b78a89034d0564008409/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/b6ddbbf400271afe4fd3c2fcde7eda6ec909cd0fd686d4368081d2cab69ba1a9/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/ec0fdd2768b9db48bd165f13a588823ba006c7afc2f8ebb426939b1f06d5dab7/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/b9f317f7838da93031288dbe3d4afc66ca2e03508db849323972aa7f0856950e/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/863ddc554ffbef3c9e239371468bfcddcb322a840c68a2ce9cc62e3a6af639c8/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/4c2be08350daddc70b1d205424be288eaa13d1f7da1f93b59477678d2bf57db5/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/da1d80e2a3f1803695cb8f68d92634db06801181b3c733a39813c7467e06ebf5/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/f9369c35a9490ba6c5c29630484614dcce6eb06bc55ee76764fe355873547236/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/dde94206dc6a32ea6a9fd33e786a0872934a2af79cac3133b257a5804b8fa4a3/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/c3b619e51c0f066442b3d27ff16be4fdaf58c6af7ed88f88bcb01e593a5791e3/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/9d42d3e207f25c60e9ddc7670d69b5da3b4b83c779dfa5b44a02318c1d4acdb3/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/b9bab3b5db64ccc9f96d0d3bd8ba3612d7d2245a5c22ca6b070dfd873ee713d7/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/0d36b199c8cc1bd5ac17a68d0f0f5d355ccf8206444967c3c60dfb26877fd668/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b49700fad70b7e612de8fa57c04708d4ab39c0464ab22e1133fb3d64ee6f18a8/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/53684eca51c164fa129f7ea66bc4fe73fd9f45f0ec705019a7bf6c31a25767f2/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/08c6283b2562a63423c6f25ff1485649342037ad75432f9c1235990a1bed9465/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-207:{mountpoint:/var/lib/containers/storage/overlay/4f147d4d297adf78f3d03c57489bb8d1c3d2bf7e1e22e75e7353c7f25fb4a866/merged major:0 minor:207 fsType:overlay blockSize:0} overlay_0-209:{mountpoint:/var/lib/containers/storage/overlay/70f1644a85f4711794be217910f98233d22fa6f652c9e68c2007762d36a503a2/merged major:0 minor:209 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/511d966e8bc82268449d19af9abafd1d68adaf2eaa5a9aa5a479baf26c6fd315/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/13b1aa9a090ca51ef65ea11c467a5c6187eb20dde0ed68f1e7e8ccdf0b9f89fe/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/6ae321d4dbc46a74d1d77aa3b2779b8834897cd37dffec3680b76d4ddb611506/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/2863cf7fe6bc77ac55dd725ab931007f026f46268c69a7176a3a6c3ec62b4f3d/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/5d61fedff6176cc446200a4fa35a87a8f7167842b88315f87e5c6ffba2cf9f24/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/b4fa931a09dc486184d64491fb2a11bc06d9b2d02d8c1f17460d21c9881c0e02/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/dfa6e1d7e6985ee5743d739b4c88c00489dcc004460f9273e93bfc5146298390/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/05863962fd0ac39e246db90ab558eebf174aecf548f5fd9e8ff41ac297a87296/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/563e81c78f03017b8ba3d19b93a85f4783441df4d74023be0b061c5720947790/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-75:{mountpoint:/var/lib/containers/storage/overlay/10beac34e76b033d17752087386d4e2b6d50ec69028372ff96cea632a8c6c39b/merged major:0 minor:75 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/ff2c55f6be31745b58ab8785cdaaacb0de38af5ec08f7848abf5dc52e3584836/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/b0100ac4d938a6ae0d7612c0931547a7fecd2b4cc4bff62ce7663751f8f149e2/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/d239b07b04126dfcd1a39ce86b44b470aaa182fa6ac155d8b62ca837972fcb3b/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/02060e03db09f53beb5e28732850f4e4ab53b6f177ebcb64fadd320548f52ac7/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/4d64a7a467f300ee1ca8a4ee8e8bb2a9448f22ce135990725184f277b600a876/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/1cde914c415f52e9c48f7d6b163fe76df7ff56a3d48a781f963d4b2c7fb3cfd7/merged major:0 minor:98 fsType:overlay blockSize:0}]
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.417930    4434 manager.go:217] Machine: {Timestamp:2026-11-04 00:57:11.41745519 +0000 UTC m=+0.079595242 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098646 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ad30b0441188ffcf7a8733bddce3a SystemUUID:ec2ad30b-0441-188f-fcf7-a8733bddce3a BootID:c80544fb-1225-444e-ad04-22e8282c30b3 Filesystems:[{Device:/var/lib/kubelet/pods/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163/volumes/kubernetes.io~projected/kube-api-access-pmbpl DeviceMajor:0 DeviceMinor:67 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2906d6c1b9398eed8f1fba8e0eea7a1eff73421702435404953f4a2f78bc40a2/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8fdadaafa855013297612924eda98564647ea4e9ea0d62e86dbf12180c9bc275/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a3ecba1b-5ec4-41f0-b49a-9d8167d31114/volumes/kubernetes.io~projected/kube-api-access-lp6xp DeviceMajor:0 DeviceMinor:201 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/b2fa4fe0-3667-44c4-963c-4572a330d29c/volumes/kubernetes.io~projected/kube-api-access-wm766 DeviceMajor:0 DeviceMinor:57 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a009b723ec156df827193e458b02dc51b8101087e6b30fbe7d6fa8c7c623a584/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/352db7a3-daed-4f66-81db-7054c082d043/volumes/kubernetes.io~projected/kube-api-access-bf2sc DeviceMajor:0 DeviceMinor:58 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/acabce609583cc4c692453e166baa34e423036bf4ee36721dd29e804ffeb4923/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7268a49bd3737eeaa66fb469259a717b2400e3892b135e3e36b337ca999b8fbd/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/5f018fcc-7d38-494c-bfe5-67dc725ec5a7/volumes/kubernetes.io~projected/kube-api-access-qt6xz DeviceMajor:0 DeviceMinor:62 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c70d7543fbc6ec5f011a642eac2d83f49a493500d047589265dc60d23f9e11ce/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/de508010-fa70-4f37-a691-bdecaecdd3e7/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/430d815c6b324559734ab1321b75e12d23ab002723ddc3880bf3dfabdd8625b5/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a3ecba1b-5ec4-41f0-b49a-9d8167d31114/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4c9ed5d4cac516d1efa4dc51e9984cd73fa80c3191d28b3fddf8deda5ac6eb6f/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/251c150b-e0d4-455b-8cdd-6a88e2fa0e48/volumes/kubernetes.io~projected/kube-api-access-vls2r DeviceMajor:0 DeviceMinor:211 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~projected/kube-api-access-25ppk DeviceMajor:0 DeviceMinor:66 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/71d0f11ae5373518cb7be62ed84520c09abcd1a497cadf6f2c64a44045c7bc48/userdata/shm DeviceMajor:0 DeviceMinor:73 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/590feae8-76fd-4069-95b5-fd8b213b5471/volumes/kubernetes.io~projected/kube-api-access-x64x6 DeviceMajor:0 DeviceMinor:197 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d34f4e73-e682-4ad2-acdf-ff6209283ace/volumes/kubernetes.io~projected/kube-api-access-kjx6z DeviceMajor:0 DeviceMinor:204 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/4813a1fc-7a03-487f-b9fa-8d1a7250768f/volumes/kubernetes.io~projected/kube-api-access-wrp79 DeviceMajor:0 DeviceMinor:61 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:64 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fcef77c15ba58adc95eff9ed46f7e23b2432c1aed48564c63294df5ce611fabc/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-75 DeviceMajor:0 DeviceMinor:75 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-207 DeviceMajor:0 DeviceMinor:207 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-209 DeviceMajor:0 DeviceMinor:209 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c5dca767-45ed-4760-9d96-d2db34951608/volumes/kubernetes.io~projected/kube-api-access-6qvnz DeviceMajor:0 DeviceMinor:60 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d41414c4-11d2-4e45-aa39-8f33726a40fd/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:214 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/728c5532-b88d-4ccc-bb31-a0278f5f84f8/volumes/kubernetes.io~projected/kube-api-access-7l7cg DeviceMajor:0 DeviceMinor:221 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/85125fc3-99ec-427f-84a5-910d207e22cd/volumes/kubernetes.io~projected/kube-api-access-gggw2 DeviceMajor:0 DeviceMinor:63 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:65 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c883a9247705974541a5312acc927dc92152e188525260213c766bc13f83fa7/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/ef3c5a66-a065-45ac-8979-a13b2e382041/volumes/kubernetes.io~projected/kube-api-access-5dbxh DeviceMajor:0 DeviceMinor:59 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c6ee6da4ff1ff1c15047a5b6a8701f84408e4a2a3d70d8c1efb2c5aef970d00/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0c6ee6da4ff1ff1 MacAddress:2a:38:10:32:df:00 Speed:10000 Mtu:8901} {Name:0c883a924770597 MacAddress:aa:7b:50:0d:d5:b8 Speed:10000 Mtu:8901} {Name:2906d6c1b9398ee MacAddress:3a:29:6c:f9:d5:c7 Speed:10000 Mtu:8901} {Name:430d815c6b32455 MacAddress:96:2c:a4:67:e6:d3 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:f0:d0:96:fb Speed:0 Mtu:9001} {Name:br-int MacAddress:ea:df:47:e9:15:00 Speed:0 Mtu:8901} {Name:ens5 MacAddress:06:ff:f0:d0:96:fb Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:52:a9:3b:5a:fa:63 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:22:e8:8f:17:c1:3d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418163    4434 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418202    4434 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418550    4434 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418576    4434 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-128.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418783    4434 topology_manager.go:143] "Creating topology manager with none policy"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418796    4434 container_manager_linux.go:311] "Creating device plugin manager"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418813    4434 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418839    4434 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.418955    4434 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419239    4434 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419378    4434 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419420    4434 kubelet.go:493] "Attempting to sync node with API server"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419439    4434 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419452    4434 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419464    4434 kubelet.go:405] "Adding apiserver pod source"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.419475    4434 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.420223    4434 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.420237    4434 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.420720    4434 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422453    4434 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422463    4434 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422537    4434 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422721    4434 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422748    4434 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422840    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422851    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422858    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422863    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422869    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422874    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422880    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422885    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422892    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422898    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422906    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422915    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422938    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 04 00:57:11.424143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.422943    4434 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 04 00:57:11.426004 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.425991    4434 server.go:1263] "Started kubelet"
Nov 04 00:57:11.426379 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.426080    4434 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 04 00:57:11.426379 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.426106    4434 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 04 00:57:11.426379 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.426145    4434 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 04 00:57:11.426554 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.426454    4434 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 04 00:57:11.426542 ip-10-0-136-128 systemd[1]: Started Kubernetes Kubelet.
Nov 04 00:57:11.427286 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.427266    4434 server.go:317] "Adding debug handlers to kubelet server"
Nov 04 00:57:11.428316 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428299    4434 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 04 00:57:11.428396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428323    4434 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 04 00:57:11.428396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428388    4434 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 04 00:57:11.428396 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428397    4434 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 04 00:57:11.428529 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428460    4434 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 04 00:57:11.428529 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428369    4434 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 00:51:32 +0000 UTC" deadline="2028-05-13 10:59:56.455989538 +0000 UTC"
Nov 04 00:57:11.428529 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.428477    4434 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13354h2m45.02751539s"
Nov 04 00:57:11.429476 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.429444    4434 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:57:11.429912 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.429897    4434 factory.go:55] Registering systemd factory
Nov 04 00:57:11.430015 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.429916    4434 factory.go:223] Registration of the systemd container factory successfully
Nov 04 00:57:11.430015 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.429942    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2fa4fe0-3667-44c4-963c-4572a330d29c" volumeName="kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 04 00:57:11.430015 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.429985    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="352db7a3-daed-4f66-81db-7054c082d043" volumeName="kubernetes.io/projected/352db7a3-daed-4f66-81db-7054c082d043-kube-api-access-bf2sc" seLinuxMountContext=""
Nov 04 00:57:11.430015 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.429999    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="590feae8-76fd-4069-95b5-fd8b213b5471" volumeName="kubernetes.io/configmap/590feae8-76fd-4069-95b5-fd8b213b5471-config-volume" seLinuxMountContext=""
Nov 04 00:57:11.430015 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430011    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f018fcc-7d38-494c-bfe5-67dc725ec5a7" volumeName="kubernetes.io/projected/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-kube-api-access-qt6xz" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430024    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb" volumeName="kubernetes.io/projected/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-kube-api-access-25ppk" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430036    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef3c5a66-a065-45ac-8979-a13b2e382041" volumeName="kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-cni-binary-copy" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430049    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef3c5a66-a065-45ac-8979-a13b2e382041" volumeName="kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-daemon-config" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430061    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="251c150b-e0d4-455b-8cdd-6a88e2fa0e48" volumeName="kubernetes.io/projected/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-api-access-vls2r" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430074    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4813a1fc-7a03-487f-b9fa-8d1a7250768f" volumeName="kubernetes.io/projected/4813a1fc-7a03-487f-b9fa-8d1a7250768f-kube-api-access-wrp79" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430085    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="590feae8-76fd-4069-95b5-fd8b213b5471" volumeName="kubernetes.io/projected/590feae8-76fd-4069-95b5-fd8b213b5471-kube-api-access-x64x6" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430097    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="728c5532-b88d-4ccc-bb31-a0278f5f84f8" volumeName="kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430109    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85125fc3-99ec-427f-84a5-910d207e22cd" volumeName="kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-script-lib" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430121    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2fa4fe0-3667-44c4-963c-4572a330d29c" volumeName="kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430133    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163" volumeName="kubernetes.io/projected/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kube-api-access-pmbpl" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430146    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="251c150b-e0d4-455b-8cdd-6a88e2fa0e48" volumeName="kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 04 00:57:11.430222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430157    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d34f4e73-e682-4ad2-acdf-ff6209283ace" volumeName="kubernetes.io/projected/d34f4e73-e682-4ad2-acdf-ff6209283ace-kube-api-access-kjx6z" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430168    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef3c5a66-a065-45ac-8979-a13b2e382041" volumeName="kubernetes.io/projected/ef3c5a66-a065-45ac-8979-a13b2e382041-kube-api-access-5dbxh" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430404    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="251c150b-e0d4-455b-8cdd-6a88e2fa0e48" volumeName="kubernetes.io/empty-dir/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-data-volume" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430427    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4813a1fc-7a03-487f-b9fa-8d1a7250768f" volumeName="kubernetes.io/configmap/4813a1fc-7a03-487f-b9fa-8d1a7250768f-iptables-alerter-script" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430452    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85125fc3-99ec-427f-84a5-910d207e22cd" volumeName="kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-env-overrides" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430466    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de508010-fa70-4f37-a691-bdecaecdd3e7" volumeName="kubernetes.io/configmap/de508010-fa70-4f37-a691-bdecaecdd3e7-konnectivity-ca" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430486    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="352db7a3-daed-4f66-81db-7054c082d043" volumeName="kubernetes.io/empty-dir/352db7a3-daed-4f66-81db-7054c082d043-tmp-dir" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430517    4434 factory.go:153] Registering CRI-O factory
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430537    4434 factory.go:223] Registration of the crio container factory successfully
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430620    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb" volumeName="kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-tmp" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430635    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85125fc3-99ec-427f-84a5-910d207e22cd" volumeName="kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-config" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430642    4434 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430655    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85125fc3-99ec-427f-84a5-910d207e22cd" volumeName="kubernetes.io/projected/85125fc3-99ec-427f-84a5-910d207e22cd-kube-api-access-gggw2" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430692    4434 factory.go:103] Registering Raw factory
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430707    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3ecba1b-5ec4-41f0-b49a-9d8167d31114" volumeName="kubernetes.io/secret/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-cert" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430730    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2fa4fe0-3667-44c4-963c-4572a330d29c" volumeName="kubernetes.io/projected/b2fa4fe0-3667-44c4-963c-4572a330d29c-kube-api-access-wm766" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430744    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f018fcc-7d38-494c-bfe5-67dc725ec5a7" volumeName="kubernetes.io/configmap/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-serviceca" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430773    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b2fa4fe0-3667-44c4-963c-4572a330d29c" volumeName="kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-binary-copy" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430787    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c5dca767-45ed-4760-9d96-d2db34951608" volumeName="kubernetes.io/projected/c5dca767-45ed-4760-9d96-d2db34951608-kube-api-access-6qvnz" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430806    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d41414c4-11d2-4e45-aa39-8f33726a40fd" volumeName="kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430822    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="590feae8-76fd-4069-95b5-fd8b213b5471" volumeName="kubernetes.io/empty-dir/590feae8-76fd-4069-95b5-fd8b213b5471-tmp-dir" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430836    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb" volumeName="kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-tuned" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430857    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3ecba1b-5ec4-41f0-b49a-9d8167d31114" volumeName="kubernetes.io/projected/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-kube-api-access-lp6xp" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430871    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de508010-fa70-4f37-a691-bdecaecdd3e7" volumeName="kubernetes.io/secret/de508010-fa70-4f37-a691-bdecaecdd3e7-agent-certs" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430890    4434 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85125fc3-99ec-427f-84a5-910d207e22cd" volumeName="kubernetes.io/secret/85125fc3-99ec-427f-84a5-910d207e22cd-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430903    4434 reconstruct.go:97] "Volume reconstruction finished"
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430912    4434 reconciler.go:29] "Reconciler: start to sync state"
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.430713    4434 manager.go:1199] Started watching for new ooms in manager
Nov 04 00:57:11.437635 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.431409    4434 manager.go:319] Starting recovery of all containers
Nov 04 00:57:11.465325 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.465283    4434 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 04 00:57:11.466702 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.466681    4434 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 04 00:57:11.466702 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.466703    4434 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 04 00:57:11.466863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.466725    4434 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 04 00:57:11.466863 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.466764    4434 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 04 00:57:11.474659 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.474635    4434 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="4548475b49e87977f600c40517a724d92559ad9a7f0cfee297ffc9404796a632" exitCode=0
Nov 04 00:57:11.474741 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.474657    4434 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="fe701ee4c40ec837a2820badf8b2b8d3bcbfb0a014a315291ab7db28e8a49eda" exitCode=0
Nov 04 00:57:11.474741 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.474684    4434 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="23cee312cdb39e346c3274253fd4fb74962d4922feef44a0769dc16cfa2679b8" exitCode=0
Nov 04 00:57:11.474741 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.474695    4434 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="5b9bc28ac2167a7e2e9b53b69a7ae0dacc4f0dad396ed4bd9563e14b0f406726" exitCode=0
Nov 04 00:57:11.474741 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.474702    4434 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="0a785a25f01ed2d41991956c6389fe105c232344e9d2f64d21fabb1793c8e522" exitCode=0
Nov 04 00:57:11.474741 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.474709    4434 generic.go:356] "Generic (PLEG): container finished" podID="b2fa4fe0-3667-44c4-963c-4572a330d29c" containerID="1ba66e20db540e81ba625445af1dd32fd67d2c405348baae4b69fc3b0ddca395" exitCode=0
Nov 04 00:57:11.486077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.486054    4434 generic.go:356] "Generic (PLEG): container finished" podID="929e66bb82351eba2f21f01154393561" containerID="27107775c6b7175b71bcafbc595148fe6b9af505a587f8fbf300de6ccd5a8128" exitCode=0
Nov 04 00:57:11.498813 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.498790    4434 manager.go:324] Recovery completed
Nov 04 00:57:11.511327 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.511309    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.512310 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.512294    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.512371 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.512323    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.512371 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.512333    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.513529 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513514    4434 cpu_manager.go:226] "Starting" policy="none"
Nov 04 00:57:11.513529 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513527    4434 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 04 00:57:11.513615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513531    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.513615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513543    4434 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 04 00:57:11.513693 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513651    4434 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 04 00:57:11.513693 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513680    4434 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 04 00:57:11.513756 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513700    4434 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 04 00:57:11.513756 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513708    4434 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 04 00:57:11.513756 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513713    4434 policy_none.go:50] "Start"
Nov 04 00:57:11.513756 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513726    4434 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 04 00:57:11.513756 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513735    4434 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 04 00:57:11.513905 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513891    4434 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 04 00:57:11.513934 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513907    4434 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 04 00:57:11.513934 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.513915    4434 policy_none.go:44] "Start"
Nov 04 00:57:11.514310 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.514295    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.514363 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.514322    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.514363 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.514332    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.517815 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.517791    4434 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 04 00:57:11.521136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521124    4434 manager.go:341] "Starting Device Plugin manager"
Nov 04 00:57:11.521203 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.521147    4434 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 04 00:57:11.521203 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521155    4434 server.go:87] "Starting device plugin registration server"
Nov 04 00:57:11.521441 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521431    4434 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 04 00:57:11.521470 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521443    4434 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 04 00:57:11.521558 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521536    4434 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 04 00:57:11.521691 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521636    4434 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 04 00:57:11.521691 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521688    4434 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 04 00:57:11.521799 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521695    4434 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 04 00:57:11.521799 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.521731    4434 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T00:57:11.521678074Z","UUID":"2421e879-a211-4393-8e82-55ec2d24e2b3","Handler":null,"Name":"","Endpoint":""}
Nov 04 00:57:11.522941 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.522873    4434 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 04 00:57:11.522941 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.522912    4434 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-128.us-west-2.compute.internal\" not found"
Nov 04 00:57:11.523084 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.523066    4434 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 04 00:57:11.523084 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.523084    4434 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 04 00:57:11.568175 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.567694    4434 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"]
Nov 04 00:57:11.568267 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.568251    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.569291 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.569274    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.569369 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.569301    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.569369 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.569315    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.569549 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.569538    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.569703 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.569690    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.570291 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.570267    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.570362 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.570300    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.570362 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.570312    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.570362 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.570275    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.570468 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.570375    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.570468 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.570388    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.570789 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.570768    4434 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.571255 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.571205    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"1d4b0eb194eb22fd85d7942f84eead7e","Type":"ContainerStarted","Data":"3c3bb287493162c8f65080df6553f29b14fc2777717ff30e357133f2f792bb99"}
Nov 04 00:57:11.571318 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.571270    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"1d4b0eb194eb22fd85d7942f84eead7e","Type":"ContainerStarted","Data":"b64a303c87efdc169db6be30eb28874348c2524e225ba37d7e7c225c7cbfffd4"}
Nov 04 00:57:11.571318 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.571297    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"929e66bb82351eba2f21f01154393561","Type":"ContainerStarted","Data":"fe5da2a38472787b10f23281d280d8fb5364914dc23048cee73729e7f5d6138b"}
Nov 04 00:57:11.571416 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.571316    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"929e66bb82351eba2f21f01154393561","Type":"ContainerDied","Data":"27107775c6b7175b71bcafbc595148fe6b9af505a587f8fbf300de6ccd5a8128"}
Nov 04 00:57:11.571416 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.571341    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal" event={"ID":"929e66bb82351eba2f21f01154393561","Type":"ContainerStarted","Data":"81929aba2471ad1126b9aed01f151b07eac94bb43e32e688b5b7c8ff5d1856af"}
Nov 04 00:57:11.571518 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.571504    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.572697 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.572660    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.572870 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.572820    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.572971 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.572898    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.573323 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.573307    4434 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-128.us-west-2.compute.internal\" not found" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.622025 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.622006    4434 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 04 00:57:11.622785 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.622770    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Nov 04 00:57:11.622859 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.622800    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 04 00:57:11.622859 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.622811    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeHasSufficientPID"
Nov 04 00:57:11.622859 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.622825    4434 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.636102 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.636085    4434 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 04 00:57:11.677775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.677752    4434 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:57:11.677858 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.677782    4434 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:57:11.679458 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.679443    4434 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:57:11.686756 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.686740    4434 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 04 00:57:11.730388 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.730373    4434 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.736555 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.736536    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d4b0eb194eb22fd85d7942f84eead7e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"1d4b0eb194eb22fd85d7942f84eead7e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.736615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.736561    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.736615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.736579    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.738927 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.738914    4434 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.738978 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.738966    4434 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.739385 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.739373    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeNotReady"
Nov 04 00:57:11.739421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.739400    4434 setters.go:546] "Node became not ready" node="ip-10-0-136-128.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T00:57:11Z","lastTransitionTime":"2026-11-04T00:57:11Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 04 00:57:11.780402 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.780347    4434 nodeinfomanager.go:471] existing CSINode "ip-10-0-136-128.us-west-2.compute.internal" is owned by different node (oldNodeID="0f81b2e7-f9e3-4d26-852b-054d1981edc5", newNodeID=""), cleaning up...
Nov 04 00:57:11.789250 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.789235    4434 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 00:57:11.789300 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.789269    4434 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.789300 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.789280    4434 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.837529 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.837510    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d4b0eb194eb22fd85d7942f84eead7e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"1d4b0eb194eb22fd85d7942f84eead7e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.837611 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.837533    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.837611 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.837550    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.837611 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.837595    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.837719 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.837607    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1d4b0eb194eb22fd85d7942f84eead7e-config\") pod \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"1d4b0eb194eb22fd85d7942f84eead7e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.837719 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.837617    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/929e66bb82351eba2f21f01154393561-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" (UID: \"929e66bb82351eba2f21f01154393561\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:11.847234 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:11.847219    4434 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 00:57:11.847290 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:11.847248    4434 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:12.420421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.420394    4434 apiserver.go:52] "Watching apiserver"
Nov 04 00:57:12.502129 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.502107    4434 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:12.526592 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.526572    4434 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 04 00:57:12.526977 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.526959    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-zb7k2","openshift-cluster-node-tuning-operator/tuned-wrcbh","kube-system/konnectivity-agent-456n2","openshift-ingress-canary/ingress-canary-qmqjk","openshift-insights/insights-runtime-extractor-26d2d","openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx","openshift-multus/multus-htt4g","openshift-image-registry/node-ca-6sgf4","openshift-dns/node-resolver-rc99q","openshift-ovn-kubernetes/ovnkube-node-w557m","kube-system/kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal","openshift-multus/network-metrics-daemon-7vlfq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27","openshift-network-operator/iptables-alerter-jv4jd","openshift-network-diagnostics/network-check-target-frjrl","openshift-dns/dns-default-5hk4x","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal","kube-system/global-pull-secret-syncer-rcwm8"]
Nov 04 00:57:12.533678 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.533648    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.534008 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.533993    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.535071 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.535052    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.541446 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541427    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-kubelet-config\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.541506 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541453    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-socket-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.541506 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541468    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-registration-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.541506 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541483    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-device-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.541587 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541531    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.541625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541579    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-sys-fs\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.541625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541607    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.541707 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541624    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:12.541707 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541644    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-cnibin\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.541707 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541680    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.541806 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541745    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.541806 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541772    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.541899 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541811    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.541899 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541858    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de508010-fa70-4f37-a691-bdecaecdd3e7-agent-certs\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:57:12.541899 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541891    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de508010-fa70-4f37-a691-bdecaecdd3e7-konnectivity-ca\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:57:12.542036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541919    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-dbus\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.542036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541946    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmbpl\" (UniqueName: \"kubernetes.io/projected/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kube-api-access-pmbpl\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.542036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.541973    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-system-cni-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.542036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.542000    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.542036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.542027    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm766\" (UniqueName: \"kubernetes.io/projected/b2fa4fe0-3667-44c4-963c-4572a330d29c-kube-api-access-wm766\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.561626 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.561607    4434 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 04 00:57:12.561707 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:12.561645    4434 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-128.us-west-2.compute.internal"
Nov 04 00:57:12.579870 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.579844    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 04 00:57:12.579964 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.579884    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2w2b9\""
Nov 04 00:57:12.580063 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.580026    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.580116 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.580093    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.580996 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.580982    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 04 00:57:12.582251 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.582230    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de508010-fa70-4f37-a691-bdecaecdd3e7-agent-certs\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:57:12.609507 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.609483    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmbpl\" (UniqueName: \"kubernetes.io/projected/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kube-api-access-pmbpl\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.629277 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.629257    4434 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 04 00:57:12.642506 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642487    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-conf-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.642564 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642513    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-netns\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.642564 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642530    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-bin\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.642564 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642548    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/590feae8-76fd-4069-95b5-fd8b213b5471-tmp-dir\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.642740 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642605    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qvnz\" (UniqueName: \"kubernetes.io/projected/c5dca767-45ed-4760-9d96-d2db34951608-kube-api-access-6qvnz\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.642740 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642684    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-sys\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.642740 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642709    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-var-lib-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.642740 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642726    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/590feae8-76fd-4069-95b5-fd8b213b5471-tmp-dir\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.642873 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642731    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-config\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.642873 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642825    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.642873 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642849    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjx6z\" (UniqueName: \"kubernetes.io/projected/d34f4e73-e682-4ad2-acdf-ff6209283ace-kube-api-access-kjx6z\") pod \"migrator-66df5d496b-mm5vx\" (UID: \"d34f4e73-e682-4ad2-acdf-ff6209283ace\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:12.642967 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642875    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-daemon-config\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.642967 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642918    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6xz\" (UniqueName: \"kubernetes.io/projected/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-kube-api-access-qt6xz\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.642967 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642947    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-modprobe-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.642967 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.642965    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.643133 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643007    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4813a1fc-7a03-487f-b9fa-8d1a7250768f-iptables-alerter-script\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.643133 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643066    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-registration-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.643133 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643096    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.643133 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643115    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-cert\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:12.643305 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643174    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-cnibin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.643305 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643188    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-etc-selinux\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.643305 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643219    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-registration-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.643305 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643227    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-serviceca\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.643305 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643263    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-var-lib-kubelet\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.643523 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643327    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-ovn\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.643523 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643350    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gggw2\" (UniqueName: \"kubernetes.io/projected/85125fc3-99ec-427f-84a5-910d207e22cd-kube-api-access-gggw2\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.643523 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643393    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25ppk\" (UniqueName: \"kubernetes.io/projected/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-kube-api-access-25ppk\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.643523 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643448    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.643523 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643490    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-data-volume\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.643523 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643520    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643547    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643557    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-data-volume\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643575    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-netns\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643614    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-kubelet\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643620    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643636    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-node-log\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643703    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-system-cni-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643745    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-kubelet-config\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643778    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp6xp\" (UniqueName: \"kubernetes.io/projected/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-kube-api-access-lp6xp\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643798    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-system-cni-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643816    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-kubelet-config\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.643825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643816    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-host\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643855    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-kubernetes\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643886    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-tuned\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643946    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-tuned\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643958    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-socket-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643985    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-device-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.643992    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-socket-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644013    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644027    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-device-dir\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644032    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysconfig\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644048    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-tmp\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644074    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/352db7a3-daed-4f66-81db-7054c082d043-hosts-file\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644107    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-kubelet\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644166    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-systemd-units\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644167    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-tmp\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644191    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-cnibin\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644241    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-cnibin\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644280    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-cni-binary-copy\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644330    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-k8s-cni-cncf-io\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644356    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644371    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-systemd\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644386    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-lib-modules\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644410    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85125fc3-99ec-427f-84a5-910d207e22cd-ovn-node-metrics-cert\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644438    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-dbus\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644477    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-socket-dir-parent\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644491    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-run\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644515    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644528    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d41414c4-11d2-4e45-aa39-8f33726a40fd-dbus\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644530    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-bin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644559    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-multus\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644582    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644600    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-script-lib\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644627    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4813a1fc-7a03-487f-b9fa-8d1a7250768f-host-slash\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644659    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrp79\" (UniqueName: \"kubernetes.io/projected/4813a1fc-7a03-487f-b9fa-8d1a7250768f-kube-api-access-wrp79\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644772    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-sys-fs\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644799    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-system-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644821    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-hostroot\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644851    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-host\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644879    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-systemd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.644872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644879    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163-sys-fs\") pod \"aws-ebs-csi-driver-node-p2c27\" (UID: \"ebfcc3b0-dbd0-42b0-b1f2-60f7d3331163\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p2c27"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644907    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-etc-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644942    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644966    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-slash\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644977    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b2fa4fe0-3667-44c4-963c-4572a330d29c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.644992    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-netd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645012    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/590feae8-76fd-4069-95b5-fd8b213b5471-config-volume\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645037    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x64x6\" (UniqueName: \"kubernetes.io/projected/590feae8-76fd-4069-95b5-fd8b213b5471-kube-api-access-x64x6\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645068    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-conf\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645173    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-multus-certs\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645210    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dbxh\" (UniqueName: \"kubernetes.io/projected/ef3c5a66-a065-45ac-8979-a13b2e382041-kube-api-access-5dbxh\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645230    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352db7a3-daed-4f66-81db-7054c082d043-tmp-dir\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645287    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf2sc\" (UniqueName: \"kubernetes.io/projected/352db7a3-daed-4f66-81db-7054c082d043-kube-api-access-bf2sc\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645338    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/352db7a3-daed-4f66-81db-7054c082d043-tmp-dir\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645374    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-log-socket\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645390    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-crio-socket\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645413    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-env-overrides\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645432    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vls2r\" (UniqueName: \"kubernetes.io/projected/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-api-access-vls2r\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645492    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.645744 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.645510    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-etc-kubernetes\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.680794 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.680731    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.680886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.680793    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.680886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.680813    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.680886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.680841    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.680886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.680869    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.680886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.680879    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.681132 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681057    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 04 00:57:12.681132 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681090    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.681227 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681141    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 04 00:57:12.681227 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681142    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.681497 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681475    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 04 00:57:12.681561 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681550    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 04 00:57:12.681615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681551    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-xq5nc\""
Nov 04 00:57:12.681615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681585    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 04 00:57:12.681615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681596    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.681615 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681608    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681621    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681621    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6tf45\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681684    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681722    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-66h22\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681748    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681786    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 04 00:57:12.681839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.681803    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ql7fx\""
Nov 04 00:57:12.682150 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682130    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.682150 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682136    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.682256 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682190    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4d9rj\""
Nov 04 00:57:12.682256 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682191    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 04 00:57:12.682256 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682224    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 04 00:57:12.682256 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682237    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de508010-fa70-4f37-a691-bdecaecdd3e7-konnectivity-ca\") pod \"konnectivity-agent-456n2\" (UID: \"de508010-fa70-4f37-a691-bdecaecdd3e7\") " pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682245    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682268    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682295    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682303    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682308    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8chh7\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682304    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682346    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6gr6q\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682350    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-47hrz\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682351    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682352    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b7knl\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682347    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682386    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.682424 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682388    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 04 00:57:12.682775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682460    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tdv96\""
Nov 04 00:57:12.682775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682497    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.682775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682559    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.682775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.682600    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 04 00:57:12.683678 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683642    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-config\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.683749 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683687    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-cert\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:12.683749 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683720    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-serviceca\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.683749 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683739    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 04 00:57:12.683886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683763    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-daemon-config\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.683886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683794    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.683886 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.683819    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4813a1fc-7a03-487f-b9fa-8d1a7250768f-iptables-alerter-script\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.684127 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.684114    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fh9wk\""
Nov 04 00:57:12.685001 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.684971    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-ovnkube-script-lib\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.685073 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.684989    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ef3c5a66-a065-45ac-8979-a13b2e382041-cni-binary-copy\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.685131 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685093    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-g2z88\""
Nov 04 00:57:12.685131 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685113    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 04 00:57:12.685192 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685156    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/85125fc3-99ec-427f-84a5-910d207e22cd-ovn-node-metrics-cert\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.685323 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685310    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/590feae8-76fd-4069-95b5-fd8b213b5471-config-volume\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.685393 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685378    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 04 00:57:12.685393 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685386    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-p4bs4\""
Nov 04 00:57:12.685468 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685380    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 04 00:57:12.685468 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685411    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-qxmlz\""
Nov 04 00:57:12.685468 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685466    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gczmn\""
Nov 04 00:57:12.685592 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.685580    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/85125fc3-99ec-427f-84a5-910d207e22cd-env-overrides\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.690032 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.690016    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm766\" (UniqueName: \"kubernetes.io/projected/b2fa4fe0-3667-44c4-963c-4572a330d29c-kube-api-access-wm766\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.690138 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.690123    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l7cg\" (UniqueName: \"kubernetes.io/projected/728c5532-b88d-4ccc-bb31-a0278f5f84f8-kube-api-access-7l7cg\") pod \"network-check-target-frjrl\" (UID: \"728c5532-b88d-4ccc-bb31-a0278f5f84f8\") " pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:12.691906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.691888    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d41414c4-11d2-4e45-aa39-8f33726a40fd-original-pull-secret\") pod \"global-pull-secret-syncer-rcwm8\" (UID: \"d41414c4-11d2-4e45-aa39-8f33726a40fd\") " pod="kube-system/global-pull-secret-syncer-rcwm8"
Nov 04 00:57:12.691957 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.691929    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-binary-copy\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.692072 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.692049    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b2fa4fe0-3667-44c4-963c-4572a330d29c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zb7k2\" (UID: \"b2fa4fe0-3667-44c4-963c-4572a330d29c\") " pod="openshift-multus/multus-additional-cni-plugins-zb7k2"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731618    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qvnz\" (UniqueName: \"kubernetes.io/projected/c5dca767-45ed-4760-9d96-d2db34951608-kube-api-access-6qvnz\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731796    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6xz\" (UniqueName: \"kubernetes.io/projected/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-kube-api-access-qt6xz\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731796    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x64x6\" (UniqueName: \"kubernetes.io/projected/590feae8-76fd-4069-95b5-fd8b213b5471-kube-api-access-x64x6\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731829    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gggw2\" (UniqueName: \"kubernetes.io/projected/85125fc3-99ec-427f-84a5-910d207e22cd-kube-api-access-gggw2\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731865    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrp79\" (UniqueName: \"kubernetes.io/projected/4813a1fc-7a03-487f-b9fa-8d1a7250768f-kube-api-access-wrp79\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731842    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dbxh\" (UniqueName: \"kubernetes.io/projected/ef3c5a66-a065-45ac-8979-a13b2e382041-kube-api-access-5dbxh\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.731906 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.731908    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vls2r\" (UniqueName: \"kubernetes.io/projected/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-kube-api-access-vls2r\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.732304 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.732167    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25ppk\" (UniqueName: \"kubernetes.io/projected/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-kube-api-access-25ppk\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.732304 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.732208    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf2sc\" (UniqueName: \"kubernetes.io/projected/352db7a3-daed-4f66-81db-7054c082d043-kube-api-access-bf2sc\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.732399 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.732349    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp6xp\" (UniqueName: \"kubernetes.io/projected/a3ecba1b-5ec4-41f0-b49a-9d8167d31114-kube-api-access-lp6xp\") pod \"ingress-canary-qmqjk\" (UID: \"a3ecba1b-5ec4-41f0-b49a-9d8167d31114\") " pod="openshift-ingress-canary/ingress-canary-qmqjk"
Nov 04 00:57:12.732399 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.732362    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjx6z\" (UniqueName: \"kubernetes.io/projected/d34f4e73-e682-4ad2-acdf-ff6209283ace-kube-api-access-kjx6z\") pod \"migrator-66df5d496b-mm5vx\" (UID: \"d34f4e73-e682-4ad2-acdf-ff6209283ace\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mm5vx"
Nov 04 00:57:12.746863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746716    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-conf-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.746863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746800    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-netns\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.746863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746821    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-conf-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.746863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746830    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-netns\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.746863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746850    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-bin\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746878    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-sys\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746908    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-var-lib-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746940    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746966    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-var-lib-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.746975    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-modprobe-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747007    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747017    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-bin\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747034    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-cnibin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747067    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-var-lib-kubelet\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747071    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-sys\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747098    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-ovn\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747136 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747127    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747160    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-netns\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747183    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-kubelet\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747211    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-node-log\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747244    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-host\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747273    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-kubernetes\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747298    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747325    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysconfig\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747355    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/352db7a3-daed-4f66-81db-7054c082d043-hosts-file\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747381    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-kubelet\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747409    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-systemd-units\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747436    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-k8s-cni-cncf-io\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747434    4434 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747465    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747526    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-systemd\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747557    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-lib-modules\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747588    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-socket-dir-parent\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.747625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747613    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-run\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747642    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747720    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-kubernetes\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747737    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747765    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-bin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747791    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysconfig\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747798    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-multus\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747827    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747860    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4813a1fc-7a03-487f-b9fa-8d1a7250768f-host-slash\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747861    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-multus\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747889    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-system-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747916    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747960    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-hostroot\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747974    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/352db7a3-daed-4f66-81db-7054c082d043-hosts-file\") pod \"node-resolver-rc99q\" (UID: \"352db7a3-daed-4f66-81db-7054c082d043\") " pod="openshift-dns/node-resolver-rc99q"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748019    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-cnibin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748028    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-kubelet\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748078    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-var-lib-kubelet\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748084    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-systemd-units\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748130    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-k8s-cni-cncf-io\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748182    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-modprobe-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748235    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-d\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748242    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-ovn\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748287    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-systemd\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.748328 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:12.748342    4434 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748372    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-lib-modules\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748429    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-multus-socket-dir-parent\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:12.748441    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:13.24841231 +0000 UTC m=+1.910552369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748479    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-run\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748488    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-netns\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748522    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748533    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-kubelet\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748571    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-node-log\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748582    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4813a1fc-7a03-487f-b9fa-8d1a7250768f-host-slash\") pod \"iptables-alerter-jv4jd\" (UID: \"4813a1fc-7a03-487f-b9fa-8d1a7250768f\") " pod="openshift-network-operator/iptables-alerter-jv4jd"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.747918    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-hostroot\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748631    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-host\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748636    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-system-cni-dir\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748659    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-systemd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748706    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-etc-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748709    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-host\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748755    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-systemd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748758    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f018fcc-7d38-494c-bfe5-67dc725ec5a7-host\") pod \"node-ca-6sgf4\" (UID: \"5f018fcc-7d38-494c-bfe5-67dc725ec5a7\") " pod="openshift-image-registry/node-ca-6sgf4"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748781    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-slash\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748811    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-netd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748815    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-run-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748838    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-conf\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748857    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-var-lib-cni-bin\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748867    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-multus-certs\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748898    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-log-socket\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748905    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-etc-openvswitch\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748927    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-crio-socket\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748957    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.748980    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-etc-kubernetes\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749032    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb-etc-sysctl-conf\") pod \"tuned-wrcbh\" (UID: \"6ebd2dcd-06f3-4aa9-83cd-f3a0a9bd7ccb\") " pod="openshift-cluster-node-tuning-operator/tuned-wrcbh"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749053    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-etc-kubernetes\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749075    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-slash\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749100    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ef3c5a66-a065-45ac-8979-a13b2e382041-host-run-multus-certs\") pod \"multus-htt4g\" (UID: \"ef3c5a66-a065-45ac-8979-a13b2e382041\") " pod="openshift-multus/multus-htt4g"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749117    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-host-cni-netd\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749145    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/85125fc3-99ec-427f-84a5-910d207e22cd-log-socket\") pod \"ovnkube-node-w557m\" (UID: \"85125fc3-99ec-427f-84a5-910d207e22cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:12.749299 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.749174    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-crio-socket\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.751512 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.751491    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/251c150b-e0d4-455b-8cdd-6a88e2fa0e48-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-26d2d\" (UID: \"251c150b-e0d4-455b-8cdd-6a88e2fa0e48\") " pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:12.752073 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.752049    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5dca767-45ed-4760-9d96-d2db34951608-metrics-certs\") pod \"network-metrics-daemon-7vlfq\" (UID: \"c5dca767-45ed-4760-9d96-d2db34951608\") " pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.833935 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.833915    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7vlfq"
Nov 04 00:57:12.835397 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:12.835377    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-26d2d"
Nov 04 00:57:13.035196 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.035163    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:13.234428 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.234352    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7vlfq"]
Nov 04 00:57:13.237922 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.237897    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:13.239107 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:13.239082    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5dca767_45ed_4760_9d96_d2db34951608.slice/crio-ce2e4630cb2f4e47c07491d6f468f541de7d4ecacc2b27eaadcad21378138870 WatchSource:0}: Error finding container ce2e4630cb2f4e47c07491d6f468f541de7d4ecacc2b27eaadcad21378138870: Status 404 returned error can't find the container with id ce2e4630cb2f4e47c07491d6f468f541de7d4ecacc2b27eaadcad21378138870
Nov 04 00:57:13.241052 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.241032    4434 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 00:57:13.252124 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.252098    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:13.252277 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:13.252259    4434 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:13.252340 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:13.252324    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:14.252306352 +0000 UTC m=+2.914446406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:13.288574 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.288557    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-26d2d"]
Nov 04 00:57:13.293561 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:13.293538    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251c150b_e0d4_455b_8cdd_6a88e2fa0e48.slice/crio-b3ea4ff6bc35a458fbcba48b3b799091f4ede01098b6334014471e18f9548812 WatchSource:0}: Error finding container b3ea4ff6bc35a458fbcba48b3b799091f4ede01098b6334014471e18f9548812: Status 404 returned error can't find the container with id b3ea4ff6bc35a458fbcba48b3b799091f4ede01098b6334014471e18f9548812
Nov 04 00:57:13.431702 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.431658    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:13.505153 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.505089    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26d2d" event={"ID":"251c150b-e0d4-455b-8cdd-6a88e2fa0e48","Type":"ContainerStarted","Data":"fcce18b17a0d3ed31433c04573a63629458e7f671e560b74c8c962548d518569"}
Nov 04 00:57:13.505256 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.505148    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26d2d" event={"ID":"251c150b-e0d4-455b-8cdd-6a88e2fa0e48","Type":"ContainerStarted","Data":"b3ea4ff6bc35a458fbcba48b3b799091f4ede01098b6334014471e18f9548812"}
Nov 04 00:57:13.506237 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.506215    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7vlfq" event={"ID":"c5dca767-45ed-4760-9d96-d2db34951608","Type":"ContainerStarted","Data":"ce2e4630cb2f4e47c07491d6f468f541de7d4ecacc2b27eaadcad21378138870"}
Nov 04 00:57:13.631355 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.631330    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:13.722688 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.722647    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:13.830356 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:13.830270    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:14.259504 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:14.259465    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:14.259649 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:14.259609    4434 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:14.259714 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:14.259690    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:16.259660267 +0000 UTC m=+4.921800321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:14.396143 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:14.396119    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-456n2"
Nov 04 00:57:15.047216 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.047184    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:15.048120 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.048091    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:15.238097 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.238072    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:15.435816 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.435741    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:15.513798 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.513761    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7vlfq" event={"ID":"c5dca767-45ed-4760-9d96-d2db34951608","Type":"ContainerStarted","Data":"d170b7f682a78596407a19a38a75160ec146e7973cbab4c69d97d5397c9fa62d"}
Nov 04 00:57:15.513988 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.513829    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7vlfq" event={"ID":"c5dca767-45ed-4760-9d96-d2db34951608","Type":"ContainerStarted","Data":"5c090da4b15f22dbee35b17c26cad80d486f745c0d024eaff4906a4e596cc1c2"}
Nov 04 00:57:15.515469 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.515441    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26d2d" event={"ID":"251c150b-e0d4-455b-8cdd-6a88e2fa0e48","Type":"ContainerStarted","Data":"fd9f9ffc9fa2774312145b6586a789c0c826f49fbd28c1a7bc29c230859067e4"}
Nov 04 00:57:15.635737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.635704    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:15.834330 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:15.834221    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:16.158017 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:16.157984    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:16.272364 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:16.272324    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:16.272552 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:16.272477    4434 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:16.272552 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:16.272544    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:20.272528843 +0000 UTC m=+8.934668901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:16.618500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:16.618426    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:16.922962 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:16.922928    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:16.923207 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:16.923177    4434 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 04 00:57:17.036898 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:17.036867    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:17.233428 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:17.233406    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:17.430162 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:17.430076    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:17.523809 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:17.523780    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-26d2d" event={"ID":"251c150b-e0d4-455b-8cdd-6a88e2fa0e48","Type":"ContainerStarted","Data":"645fee7fd29441ac5f0d52cc5a9c3e30a301338dab53d7266b5f221921eb89ff"}
Nov 04 00:57:17.640733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:17.640707    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:17.831275 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:17.831186    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:18.839781 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:18.839744    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:18.998368 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:18.998337    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:19.240220 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:19.240190    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:19.435659 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:19.435632    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:19.667723 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:19.667694    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:19.823834 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:19.823808    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:20.041979 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:20.041893    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:20.297989 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:20.297905    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:20.298142 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:20.298049    4434 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 04 00:57:20.298142 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:20.298108    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls podName:590feae8-76fd-4069-95b5-fd8b213b5471 nodeName:}" failed. No retries permitted until 2026-11-04 00:57:28.298093521 +0000 UTC m=+16.960233575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls") pod "dns-default-5hk4x" (UID: "590feae8-76fd-4069-95b5-fd8b213b5471") : secret "dns-default-metrics-tls" not found
Nov 04 00:57:20.822504 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:20.822471    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:21.034450 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:21.034417    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:21.234399 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:21.234367    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:21.437096 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:21.437056    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:21.895267 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:21.895234    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 00:57:22.170051 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:22.169980    4434 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-136-128.us-west-2.compute.internal" event="NodeReady"
Nov 04 00:57:23.313520 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:23.313490    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w557m"
Nov 04 00:57:23.790265 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:23.790232    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:23.940097 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:23.940067    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:25.578775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:25.578746    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:25.752091 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:25.752051    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:25.944470 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:25.944434    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:26.135554 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:26.135522    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:26.297158 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:26.297079    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:28.354344 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:28.354313    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:28.356782 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:28.356755    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/590feae8-76fd-4069-95b5-fd8b213b5471-metrics-tls\") pod \"dns-default-5hk4x\" (UID: \"590feae8-76fd-4069-95b5-fd8b213b5471\") " pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:28.435106 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:28.435080    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:28.806936 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:28.806901    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5hk4x"]
Nov 04 00:57:28.811386 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:28.811352    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod590feae8_76fd_4069_95b5_fd8b213b5471.slice/crio-7a28b40dd38a04a1e4304ffa6f7eba18d74bd3212149b3409de88f59857d5ab5 WatchSource:0}: Error finding container 7a28b40dd38a04a1e4304ffa6f7eba18d74bd3212149b3409de88f59857d5ab5: Status 404 returned error can't find the container with id 7a28b40dd38a04a1e4304ffa6f7eba18d74bd3212149b3409de88f59857d5ab5
Nov 04 00:57:29.472506 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:29.472436    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:29.558819 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:29.558783    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5hk4x" event={"ID":"590feae8-76fd-4069-95b5-fd8b213b5471","Type":"ContainerStarted","Data":"7a28b40dd38a04a1e4304ffa6f7eba18d74bd3212149b3409de88f59857d5ab5"}
Nov 04 00:57:29.668186 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:29.668153    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:30.159629 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:30.159596    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:30.329710 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:30.329654    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:30.491999 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:30.491907    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:30.655684 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:30.655641    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:31.224717 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.224680    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:31.288595 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.288528    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"]
Nov 04 00:57:31.290598 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.290579    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:31.343860 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.343834    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 04 00:57:31.374415 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.374384    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11cbfdab-08e2-45a8-a2c6-f0d37a871563-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-xxspl\" (UID: \"11cbfdab-08e2-45a8-a2c6-f0d37a871563\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:31.382674 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.382646    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"]
Nov 04 00:57:31.394188 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.394162    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4tsr2\""
Nov 04 00:57:31.475351 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.475329    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11cbfdab-08e2-45a8-a2c6-f0d37a871563-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-xxspl\" (UID: \"11cbfdab-08e2-45a8-a2c6-f0d37a871563\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:31.477620 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.477597    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11cbfdab-08e2-45a8-a2c6-f0d37a871563-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-xxspl\" (UID: \"11cbfdab-08e2-45a8-a2c6-f0d37a871563\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:31.486626 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.486608    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:31.568083 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.567990    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5hk4x" event={"ID":"590feae8-76fd-4069-95b5-fd8b213b5471","Type":"ContainerStarted","Data":"c3c6f0edd7e6df27d02f8349b2f80ecf499245ab7fb9213625bf5e3d5bb392f3"}
Nov 04 00:57:31.568083 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.568064    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5hk4x" event={"ID":"590feae8-76fd-4069-95b5-fd8b213b5471","Type":"ContainerStarted","Data":"99f06c03517b4c6ac775c75ff9914050333b4b6f67b8a5638652e2a8d7f0fd41"}
Nov 04 00:57:31.568600 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.568176    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:31.599443 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.599408    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:31.964845 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:31.964809    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"]
Nov 04 00:57:31.969225 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:31.969196    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cbfdab_08e2_45a8_a2c6_f0d37a871563.slice/crio-df6262edd92c7e738799515dab124b15c9d8ee88f836661fcd9020c9ed13a234 WatchSource:0}: Error finding container df6262edd92c7e738799515dab124b15c9d8ee88f836661fcd9020c9ed13a234: Status 404 returned error can't find the container with id df6262edd92c7e738799515dab124b15c9d8ee88f836661fcd9020c9ed13a234
Nov 04 00:57:32.572958 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:32.572923    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl" event={"ID":"11cbfdab-08e2-45a8-a2c6-f0d37a871563","Type":"ContainerStarted","Data":"df6262edd92c7e738799515dab124b15c9d8ee88f836661fcd9020c9ed13a234"}
Nov 04 00:57:34.580461 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:34.580369    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl" event={"ID":"11cbfdab-08e2-45a8-a2c6-f0d37a871563","Type":"ContainerStarted","Data":"73d31574a12f799426ff2e51b71bde91b771accbe53cbf25f6edae4653c177ef"}
Nov 04 00:57:34.580898 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:34.580563    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:34.585229 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:34.585210    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl"
Nov 04 00:57:34.694793 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:34.694745    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-xxspl" podStartSLOduration=1.374744572 podStartE2EDuration="3.694731285s" totalImagesPullingTime="2.319986713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:34.694089138 +0000 UTC m=+23.356229200" watchObservedRunningTime="2026-11-04 00:57:34.694731285 +0000 UTC m=+23.356871346"
Nov 04 00:57:35.522994 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:35.522962    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:35.698505 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:35.698472    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:35.865166 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:35.865092    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:36.045725 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:36.045681    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:36.774409 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:36.774373    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:37.039781 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.039703    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:37.495225 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.495194    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:37.725191 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.725161    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:37.798284 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.798254    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-xhl8w"]
Nov 04 00:57:37.804069 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.804043    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:37.861051 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.861024    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 04 00:57:37.861198 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.861079    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 04 00:57:37.911335 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.911309    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-wpj4m\""
Nov 04 00:57:37.912002 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.911977    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-xhl8w"]
Nov 04 00:57:37.914644 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.914628    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcn76\" (UniqueName: \"kubernetes.io/projected/ff8a5906-1d5b-4d35-bd99-3ed57cd09d90-kube-api-access-bcn76\") pod \"downloads-6dd789c8b7-xhl8w\" (UID: \"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90\") " pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:37.914730 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.914688    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff8a5906-1d5b-4d35-bd99-3ed57cd09d90-tmp\") pod \"downloads-6dd789c8b7-xhl8w\" (UID: \"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90\") " pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:37.967327 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:37.967290    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:38.015543 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.015512    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcn76\" (UniqueName: \"kubernetes.io/projected/ff8a5906-1d5b-4d35-bd99-3ed57cd09d90-kube-api-access-bcn76\") pod \"downloads-6dd789c8b7-xhl8w\" (UID: \"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90\") " pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:38.015735 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.015578    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff8a5906-1d5b-4d35-bd99-3ed57cd09d90-tmp\") pod \"downloads-6dd789c8b7-xhl8w\" (UID: \"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90\") " pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:38.015980 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.015958    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ff8a5906-1d5b-4d35-bd99-3ed57cd09d90-tmp\") pod \"downloads-6dd789c8b7-xhl8w\" (UID: \"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90\") " pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:38.079265 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.079193    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcn76\" (UniqueName: \"kubernetes.io/projected/ff8a5906-1d5b-4d35-bd99-3ed57cd09d90-kube-api-access-bcn76\") pod \"downloads-6dd789c8b7-xhl8w\" (UID: \"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90\") " pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:38.115195 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.115160    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:38.436426 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.436391    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:38.588971 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:38.588923    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-xhl8w"]
Nov 04 00:57:38.593303 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:38.593278    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8a5906_1d5b_4d35_bd99_3ed57cd09d90.slice/crio-9ed9db4b21a6c63f72a9778cb954e7ca2fffc6b946c435302bc7734106565ff5 WatchSource:0}: Error finding container 9ed9db4b21a6c63f72a9778cb954e7ca2fffc6b946c435302bc7734106565ff5: Status 404 returned error can't find the container with id 9ed9db4b21a6c63f72a9778cb954e7ca2fffc6b946c435302bc7734106565ff5
Nov 04 00:57:39.596924 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:39.596893    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-xhl8w" event={"ID":"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90","Type":"ContainerStarted","Data":"9ed9db4b21a6c63f72a9778cb954e7ca2fffc6b946c435302bc7734106565ff5"}
Nov 04 00:57:39.641994 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:39.641965    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:39.810863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:39.810829    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:39.986187 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:39.986151    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:41.524754 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:41.524717    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:41.575726 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:41.575699    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5hk4x"
Nov 04 00:57:41.685101 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:41.685051    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5hk4x_590feae8-76fd-4069-95b5-fd8b213b5471/dns/0.log"
Nov 04 00:57:41.788220 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:41.788128    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:41.791261 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:41.791236    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5hk4x_590feae8-76fd-4069-95b5-fd8b213b5471/kube-rbac-proxy/0.log"
Nov 04 00:57:41.962838 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:41.962804    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:42.147562 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.147534    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:42.315891 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.315856    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:42.458324 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.458244    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"]
Nov 04 00:57:42.487811 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.487784    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.500998 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.500964    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:42.519390 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.519360    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"]
Nov 04 00:57:42.534357 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.534334    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m4jls"]
Nov 04 00:57:42.545583 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.545552    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/03166a27-52e3-4e11-aeeb-d6d851cd42c4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.545694 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.545593    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03166a27-52e3-4e11-aeeb-d6d851cd42c4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.545694 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.545651    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03166a27-52e3-4e11-aeeb-d6d851cd42c4-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.545792 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.545732    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v64v\" (UniqueName: \"kubernetes.io/projected/03166a27-52e3-4e11-aeeb-d6d851cd42c4-kube-api-access-4v64v\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.555967 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.555947    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.584274 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.584249    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-qrz5z\""
Nov 04 00:57:42.584274 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.584249    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 04 00:57:42.584427 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.584249    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 04 00:57:42.584427 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.584254    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Nov 04 00:57:42.584427 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.584304    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 04 00:57:42.584427 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.584325    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Nov 04 00:57:42.591472 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.591452    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"]
Nov 04 00:57:42.606284 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.606265    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.639917 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.639896    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"]
Nov 04 00:57:42.645402 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.645359    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 04 00:57:42.645402 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.645396    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 04 00:57:42.645614 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.645499    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tjpbs\""
Nov 04 00:57:42.645614 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.645600    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 04 00:57:42.646842 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.646819    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03166a27-52e3-4e11-aeeb-d6d851cd42c4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.646928 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.646870    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ccbf275-c8e0-40d2-923a-699910bc0c53-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.646928 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.646902    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.647036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.646943    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-tls\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.646969    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed2a3230-bc57-4fe5-8a85-1487aab119c3-metrics-client-ca\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.646986    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-wtmp\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647007    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647040    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647060    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647076    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-root\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647101    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/03166a27-52e3-4e11-aeeb-d6d851cd42c4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647134    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v64v\" (UniqueName: \"kubernetes.io/projected/03166a27-52e3-4e11-aeeb-d6d851cd42c4-kube-api-access-4v64v\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647166    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8rl\" (UniqueName: \"kubernetes.io/projected/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-api-access-js8rl\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647205    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-textfile\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647231    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647624 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647381    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-sys\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.647624 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647450    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4ccbf275-c8e0-40d2-923a-699910bc0c53-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.647624 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.647537    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03166a27-52e3-4e11-aeeb-d6d851cd42c4-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.648288 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.648258    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03166a27-52e3-4e11-aeeb-d6d851cd42c4-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.648372 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.648319    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fmc7\" (UniqueName: \"kubernetes.io/projected/ed2a3230-bc57-4fe5-8a85-1487aab119c3-kube-api-access-4fmc7\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.649645 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.649627    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03166a27-52e3-4e11-aeeb-d6d851cd42c4-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.650057 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.650033    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/03166a27-52e3-4e11-aeeb-d6d851cd42c4-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.675621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.675590    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:42.692893 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.692878    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rc99q_352db7a3-daed-4f66-81db-7054c082d043/dns-node-resolver/0.log"
Nov 04 00:57:42.707201 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.707170    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v64v\" (UniqueName: \"kubernetes.io/projected/03166a27-52e3-4e11-aeeb-d6d851cd42c4-kube-api-access-4v64v\") pod \"openshift-state-metrics-7bd7489957-fzthg\" (UID: \"03166a27-52e3-4e11-aeeb-d6d851cd42c4\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.742464 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.742408    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cdkb9\""
Nov 04 00:57:42.742464 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.742408    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Nov 04 00:57:42.742622 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.742476    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Nov 04 00:57:42.742622 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.742565    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Nov 04 00:57:42.749192 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749170    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-textfile\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749207    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749369 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749310    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-sys\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749369 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749344    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4ccbf275-c8e0-40d2-923a-699910bc0c53-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.749475 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749378    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fmc7\" (UniqueName: \"kubernetes.io/projected/ed2a3230-bc57-4fe5-8a85-1487aab119c3-kube-api-access-4fmc7\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749475 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749411    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ccbf275-c8e0-40d2-923a-699910bc0c53-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.749475 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749438    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749477    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-tls\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749551    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed2a3230-bc57-4fe5-8a85-1487aab119c3-metrics-client-ca\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749583    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-wtmp\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749612    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749647    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749686    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-textfile\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749697    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.749733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749723    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-root\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.750215 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749792    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-sys\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.750215 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749841    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-root\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.750215 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.749926    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-wtmp\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.750215 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.750024    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8rl\" (UniqueName: \"kubernetes.io/projected/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-api-access-js8rl\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.750215 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.750033    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/4ccbf275-c8e0-40d2-923a-699910bc0c53-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.750466 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.750345    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.750851 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.750829    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.750982 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.750958    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ccbf275-c8e0-40d2-923a-699910bc0c53-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.751139 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.751112    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed2a3230-bc57-4fe5-8a85-1487aab119c3-metrics-client-ca\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.752775 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.752753    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.753042 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.753016    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.753221 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.753151    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed2a3230-bc57-4fe5-8a85-1487aab119c3-node-exporter-tls\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.753546 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.753525    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.812108 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.812072    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fmc7\" (UniqueName: \"kubernetes.io/projected/ed2a3230-bc57-4fe5-8a85-1487aab119c3-kube-api-access-4fmc7\") pod \"node-exporter-m4jls\" (UID: \"ed2a3230-bc57-4fe5-8a85-1487aab119c3\") " pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.812241 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.812232    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8rl\" (UniqueName: \"kubernetes.io/projected/4ccbf275-c8e0-40d2-923a-699910bc0c53-kube-api-access-js8rl\") pod \"kube-state-metrics-5dc8d7d89b-5ntrn\" (UID: \"4ccbf275-c8e0-40d2-923a-699910bc0c53\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:42.836679 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.836640    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"
Nov 04 00:57:42.905654 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.905623    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:42.906593 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.906440    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m4jls"
Nov 04 00:57:42.938438 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:42.938413    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2a3230_bc57_4fe5_8a85_1487aab119c3.slice/crio-ab96f79ca7fc4b91f078091e8d7753f0a83f4316ea09a2c3258606a7e66fed81 WatchSource:0}: Error finding container ab96f79ca7fc4b91f078091e8d7753f0a83f4316ea09a2c3258606a7e66fed81: Status 404 returned error can't find the container with id ab96f79ca7fc4b91f078091e8d7753f0a83f4316ea09a2c3258606a7e66fed81
Nov 04 00:57:42.963273 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:42.963244    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"
Nov 04 00:57:43.112734 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.112702    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:43.285025 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.284931    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg"]
Nov 04 00:57:43.289565 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:43.289536    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03166a27_52e3_4e11_aeeb_d6d851cd42c4.slice/crio-e5a4f24b6c03e91612c974f53af8ca7cc7316ca7325125936462af69c75ce7b6 WatchSource:0}: Error finding container e5a4f24b6c03e91612c974f53af8ca7cc7316ca7325125936462af69c75ce7b6: Status 404 returned error can't find the container with id e5a4f24b6c03e91612c974f53af8ca7cc7316ca7325125936462af69c75ce7b6
Nov 04 00:57:43.405462 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.405393    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn"]
Nov 04 00:57:43.409390 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:43.409367    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ccbf275_c8e0_40d2_923a_699910bc0c53.slice/crio-b8f293bb3c20e2c9a61ac7b8fddbfc0c3ef43b9bf334a9817c0f3fa367c2ff57 WatchSource:0}: Error finding container b8f293bb3c20e2c9a61ac7b8fddbfc0c3ef43b9bf334a9817c0f3fa367c2ff57: Status 404 returned error can't find the container with id b8f293bb3c20e2c9a61ac7b8fddbfc0c3ef43b9bf334a9817c0f3fa367c2ff57
Nov 04 00:57:43.611949 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.611860    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg" event={"ID":"03166a27-52e3-4e11-aeeb-d6d851cd42c4","Type":"ContainerStarted","Data":"18b5c1c11e80cc1b0338521ff955b9454b0b8bcc2488426ffd27be95ce04e71d"}
Nov 04 00:57:43.611949 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.611927    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg" event={"ID":"03166a27-52e3-4e11-aeeb-d6d851cd42c4","Type":"ContainerStarted","Data":"8d89a3474a1667ef677a82a38ce350f0883b8db0d7bc62d212d7b5a51508ce00"}
Nov 04 00:57:43.612430 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.611956    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg" event={"ID":"03166a27-52e3-4e11-aeeb-d6d851cd42c4","Type":"ContainerStarted","Data":"e5a4f24b6c03e91612c974f53af8ca7cc7316ca7325125936462af69c75ce7b6"}
Nov 04 00:57:43.613166 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.613140    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn" event={"ID":"4ccbf275-c8e0-40d2-923a-699910bc0c53","Type":"ContainerStarted","Data":"b8f293bb3c20e2c9a61ac7b8fddbfc0c3ef43b9bf334a9817c0f3fa367c2ff57"}
Nov 04 00:57:43.614403 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.614368    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4jls" event={"ID":"ed2a3230-bc57-4fe5-8a85-1487aab119c3","Type":"ContainerStarted","Data":"ab96f79ca7fc4b91f078091e8d7753f0a83f4316ea09a2c3258606a7e66fed81"}
Nov 04 00:57:43.692999 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.692973    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qmqjk_a3ecba1b-5ec4-41f0-b49a-9d8167d31114/serve-healthcheck-canary/0.log"
Nov 04 00:57:43.893242 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:43.893207    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:44.065604 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.065569    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:44.768398 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.768363    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:44.831077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.831046    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:57:44.845806 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.845780    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.902753 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.901951    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 00:57:44.902753 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.902078    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-phxps\""
Nov 04 00:57:44.902753 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.902161    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 00:57:44.902753 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.902253    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 00:57:44.952398 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.952361    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 00:57:44.952398 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.952384    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 00:57:44.967152 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967101    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967160    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967182    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967210    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967227    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967261    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967295    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x62vj\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967331    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967358    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967412    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967463    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967493    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:44.967633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:44.967548    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.002628 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.002597    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 00:57:45.003274 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.003249    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 00:57:45.003799 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.003777    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:57:45.051478 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.051415    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-frjrl"
Nov 04 00:57:45.053156 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.053131    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 00:57:45.053354 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.053335    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 00:57:45.068762 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068728    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.068868 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068768    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.068868 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068796    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.068868 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068828    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.068868 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068851    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068884    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068929    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x62vj\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068965    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.068990    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.069019    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.069052    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069414 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.069091    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.069414 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.069127    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.071273 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.070846    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.071273 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.071057    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.075131 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.073826    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.077683 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.075454    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.078692 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.078650    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.080919 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.080869    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.080919 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.080907    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.081063 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.081036    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.081436 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.081410    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.081587 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.081570    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.082263 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.082244    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.082757 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.082735    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.128782 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.128756    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x62vj\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj\") pod \"alertmanager-main-0\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.271165 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.270946    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:57:45.623191 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.623160    4434 generic.go:356] "Generic (PLEG): container finished" podID="ed2a3230-bc57-4fe5-8a85-1487aab119c3" containerID="8c2ad4e6dab3655dec57ddbc7a334dd5cbc36484344e3d7d231745df63f5d211" exitCode=0
Nov 04 00:57:45.623345 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.623199    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4jls" event={"ID":"ed2a3230-bc57-4fe5-8a85-1487aab119c3","Type":"ContainerDied","Data":"8c2ad4e6dab3655dec57ddbc7a334dd5cbc36484344e3d7d231745df63f5d211"}
Nov 04 00:57:45.732394 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:45.732284    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:57:45.863850 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:45.863812    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda034a8b0_8581_4186_84d8_573a5f049e67.slice/crio-a76b64394b0210fc5fcbe19c8a5eecd7b94d2dc7c260e2808ac3fe9019946f21 WatchSource:0}: Error finding container a76b64394b0210fc5fcbe19c8a5eecd7b94d2dc7c260e2808ac3fe9019946f21: Status 404 returned error can't find the container with id a76b64394b0210fc5fcbe19c8a5eecd7b94d2dc7c260e2808ac3fe9019946f21
Nov 04 00:57:46.584012 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.583905    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:46.630426 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.630391    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg" event={"ID":"03166a27-52e3-4e11-aeeb-d6d851cd42c4","Type":"ContainerStarted","Data":"5b576cee53393849c306dc470b5e05b41edf38345c6269c6a502e01295a51e9b"}
Nov 04 00:57:46.631776 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.631750    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"a76b64394b0210fc5fcbe19c8a5eecd7b94d2dc7c260e2808ac3fe9019946f21"}
Nov 04 00:57:46.634106 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.634076    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn" event={"ID":"4ccbf275-c8e0-40d2-923a-699910bc0c53","Type":"ContainerStarted","Data":"a74c518edc27a95dec7dbce7464309304a4cf5bb2d0f41cfc6e4baeb33e0cc8e"}
Nov 04 00:57:46.634235 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.634139    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn" event={"ID":"4ccbf275-c8e0-40d2-923a-699910bc0c53","Type":"ContainerStarted","Data":"538a9fc28fdd04898c4042c4880529866fa665cca13b447affa654b41c7e1b60"}
Nov 04 00:57:46.634235 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.634165    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn" event={"ID":"4ccbf275-c8e0-40d2-923a-699910bc0c53","Type":"ContainerStarted","Data":"330aca0ea8fb8c47ecd8361e5a8529d22e917c7927c6393fa9585782c49b47b8"}
Nov 04 00:57:46.636591 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.636568    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4jls" event={"ID":"ed2a3230-bc57-4fe5-8a85-1487aab119c3","Type":"ContainerStarted","Data":"8ab051801b8a824ed320164916dce5281e7b3703079a51228a9c760c8039ee4f"}
Nov 04 00:57:46.636690 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.636614    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m4jls" event={"ID":"ed2a3230-bc57-4fe5-8a85-1487aab119c3","Type":"ContainerStarted","Data":"8d17e4f8a64db3f344be90e431db13749cf117378d2ec654f3632cbaa87956c0"}
Nov 04 00:57:46.747937 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.747888    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-fzthg" podStartSLOduration=2.285140362 podStartE2EDuration="4.747872241s" totalImagesPullingTime="2.462731879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:46.747531357 +0000 UTC m=+35.409671419" watchObservedRunningTime="2026-11-04 00:57:46.747872241 +0000 UTC m=+35.410012280"
Nov 04 00:57:46.762094 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.762056    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:46.865310 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.865200    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m4jls" podStartSLOduration=2.5472572749999998 podStartE2EDuration="4.865181316s" totalImagesPullingTime="2.222451427s" totalInitContainerRuntime="95.472614ms" isStatefulPod=true podCreationTimestamp="2026-11-04 00:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:46.864399352 +0000 UTC m=+35.526539414" watchObservedRunningTime="2026-11-04 00:57:46.865181316 +0000 UTC m=+35.527321379"
Nov 04 00:57:46.963930 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.963892    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:46.980611 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:46.980556    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-5ntrn" podStartSLOduration=2.329295327 podStartE2EDuration="4.980540204s" totalImagesPullingTime="2.651244877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:46.979689712 +0000 UTC m=+35.641829777" watchObservedRunningTime="2026-11-04 00:57:46.980540204 +0000 UTC m=+35.642680268"
Nov 04 00:57:47.122890 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.122807    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:47.280110 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.280069    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:47.417399 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.417319    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-9958fc47b-vts5v"]
Nov 04 00:57:47.419869 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.419843    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.446711 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.446639    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:47.474956 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.474880    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Nov 04 00:57:47.475226 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.475186    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Nov 04 00:57:47.475335 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.475322    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fbpj9hmgv652t\""
Nov 04 00:57:47.475579 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.475547    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qfllr\""
Nov 04 00:57:47.475659 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.475604    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 04 00:57:47.475827 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.475762    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Nov 04 00:57:47.479378 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.479358    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9958fc47b-vts5v"]
Nov 04 00:57:47.488974 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.488941    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-audit-log\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.489085 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.488985    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b72l\" (UniqueName: \"kubernetes.io/projected/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-kube-api-access-9b72l\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.489085 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.489026    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.489085 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.489073    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-metrics-server-audit-profiles\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.489250 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.489105    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-secret-metrics-server-client-certs\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.489250 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.489133    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-secret-metrics-server-tls\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.489250 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.489171    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-client-ca-bundle\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590331 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590306    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590449 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590362    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-metrics-server-audit-profiles\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590449 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590390    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-secret-metrics-server-client-certs\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590449 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590409    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-secret-metrics-server-tls\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590449 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590445    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-client-ca-bundle\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590726 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590484    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-audit-log\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.590726 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.590507    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b72l\" (UniqueName: \"kubernetes.io/projected/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-kube-api-access-9b72l\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.591939 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.591914    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.592446 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.592245    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-metrics-server-audit-profiles\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.592942 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.592917    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-audit-log\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.594134 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.594112    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-secret-metrics-server-tls\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.594970 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.594945    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-client-ca-bundle\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.595409 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.595386    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-secret-metrics-server-client-certs\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.642301 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.642268    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e" exitCode=0
Nov 04 00:57:47.642430 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.642398    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e"}
Nov 04 00:57:47.647521 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.647494    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b72l\" (UniqueName: \"kubernetes.io/projected/cb9f1f13-14ee-4f50-80ce-dc8ada58f953-kube-api-access-9b72l\") pod \"metrics-server-9958fc47b-vts5v\" (UID: \"cb9f1f13-14ee-4f50-80ce-dc8ada58f953\") " pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.733470 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.733393    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:57:47.747400 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.747375    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:47.900807 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:47.900751    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:48.058859 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:48.058772    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:48.159248 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:48.159215    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-9958fc47b-vts5v"]
Nov 04 00:57:48.163112 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:48.163079    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9f1f13_14ee_4f50_80ce_dc8ada58f953.slice/crio-a4596147c9c14c6a9ed539c8ddfc324a8445b89552c55dae749dc702288343b0 WatchSource:0}: Error finding container a4596147c9c14c6a9ed539c8ddfc324a8445b89552c55dae749dc702288343b0: Status 404 returned error can't find the container with id a4596147c9c14c6a9ed539c8ddfc324a8445b89552c55dae749dc702288343b0
Nov 04 00:57:48.649291 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:48.649261    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v" event={"ID":"cb9f1f13-14ee-4f50-80ce-dc8ada58f953","Type":"ContainerStarted","Data":"a4596147c9c14c6a9ed539c8ddfc324a8445b89552c55dae749dc702288343b0"}
Nov 04 00:57:49.844152 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.844116    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59d57f5b7d-cfbh4"]
Nov 04 00:57:49.846576 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.846547    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.910917 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.910887    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.911055 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.910927    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.911055 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.910972    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.911055 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.911036    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.911154 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.911106    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.911154 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.911143    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zcsj\" (UniqueName: \"kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.911215 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.911167    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:49.952186 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.952129    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 04 00:57:49.952186 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.952140    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 04 00:57:49.952186 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.952184    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 04 00:57:49.952433 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.952185    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 04 00:57:49.952433 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.952249    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 04 00:57:49.952433 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.952333    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6x79p\""
Nov 04 00:57:49.953706 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:49.953370    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d57f5b7d-cfbh4"]
Nov 04 00:57:50.012433 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012400    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.012606 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012449    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.012606 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012484    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.012606 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012526    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.012606 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012579    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.013800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012615    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.013800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.012643    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zcsj\" (UniqueName: \"kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.013800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.013519    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.013800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.013618    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.013988 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.013848    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.014222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.014198    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.017077 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.017057    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.017181 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.017126    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.052221 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.052187    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:50.070236 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.070211    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zcsj\" (UniqueName: \"kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj\") pod \"console-59d57f5b7d-cfbh4\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") " pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.159854 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.159820    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:57:50.213356 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.213321    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:50.546257 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.546226    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d57f5b7d-cfbh4"]
Nov 04 00:57:50.550304 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:50.550271    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70c70c2_2de4_4fc0_8019_87277af64a37.slice/crio-40f59f86049efe88c04787dc0de043bf4d031d33dcf16b8e385b9b80b380aba2 WatchSource:0}: Error finding container 40f59f86049efe88c04787dc0de043bf4d031d33dcf16b8e385b9b80b380aba2: Status 404 returned error can't find the container with id 40f59f86049efe88c04787dc0de043bf4d031d33dcf16b8e385b9b80b380aba2
Nov 04 00:57:50.659343 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.659303    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d57f5b7d-cfbh4" event={"ID":"e70c70c2-2de4-4fc0-8019-87277af64a37","Type":"ContainerStarted","Data":"40f59f86049efe88c04787dc0de043bf4d031d33dcf16b8e385b9b80b380aba2"}
Nov 04 00:57:50.661957 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.661224    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v" event={"ID":"cb9f1f13-14ee-4f50-80ce-dc8ada58f953","Type":"ContainerStarted","Data":"048c7708f1d9de60483a9239364ec5c90781effb13185a71dd315bc1740359c0"}
Nov 04 00:57:50.778739 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:50.778628    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v" podStartSLOduration=1.5153711539999999 podStartE2EDuration="3.778608293s" totalImagesPullingTime="2.263237139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:50.778452126 +0000 UTC m=+39.440592199" watchObservedRunningTime="2026-11-04 00:57:50.778608293 +0000 UTC m=+39.440748355"
Nov 04 00:57:51.596207 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.596150    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:51.599876 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.599847    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:57:51.607766 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.607742    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.660890 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.660855    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 00:57:51.661202 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.661172    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 00:57:51.661202 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.661176    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 00:57:51.661356 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.661203    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3gjddat11co3b\""
Nov 04 00:57:51.662589 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.662569    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 00:57:51.662589 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.662582    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 00:57:51.662748 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.662599    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xpqx4\""
Nov 04 00:57:51.668381 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.668350    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511"}
Nov 04 00:57:51.668484 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.668419    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e"}
Nov 04 00:57:51.668484 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.668460    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c"}
Nov 04 00:57:51.668597 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.668502    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f"}
Nov 04 00:57:51.668597 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.668538    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c"}
Nov 04 00:57:51.711213 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711096    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 00:57:51.711213 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711188    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 00:57:51.711370 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711242    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 00:57:51.711416 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711391    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 00:57:51.711469 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711432    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 00:57:51.711469 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711452    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 00:57:51.711535 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.711500    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 00:57:51.727640 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.727577    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.727640 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.727627    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.727851 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.727659    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.727851 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.727767    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728012 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.727981    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728216 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728037    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728216 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728094    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728338 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728222    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728338 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728260    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728338 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728314    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728472 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728361    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728472 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728402    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fgxj\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728472 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728443    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728581 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728474    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728581 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728521    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728581 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728539    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728581 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728556    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728581 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728573    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728587    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.728863 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.728644    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.764648 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.764596    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 00:57:51.764769 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.764737    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 00:57:51.818620 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.818589    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:57:51.829108 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829074    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829230 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829117    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829230 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829139    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829230 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829163    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829230 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829179    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fgxj\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829230 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829205    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829230 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829228    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.829526 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829266    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829838    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829922    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829953    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.829995    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830020    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830038    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830044    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830053    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830146    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830178    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830215    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830247    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830280    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830313    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830346    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.830772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830372    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.831069 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.830914    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.831223 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.831192    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.832623 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.832602    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.833028 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.832985    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.833120 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.833085    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.833301 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.833276    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.833797 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.833776    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.834876 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.834848    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.835060 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.835026    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.835060 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.835042    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.835430 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.835410    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.835542 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.835502    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.836721 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.836697    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.836795 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.836748    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.836854 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.836812    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.887147 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.887118    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fgxj\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj\") pod \"prometheus-k8s-0\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:51.901617 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:57:51.901575    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 00:57:51.929222 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:51.929197    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:57:52.368190 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:52.368106    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:57:52.372530 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:57:52.372482    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaafe79fb_4c72_42de_b4df_a8127425187a.slice/crio-b0657f35534316731d0cae37705fea335f9d7555d33bf5cf25aa8c7d2504e3fa WatchSource:0}: Error finding container b0657f35534316731d0cae37705fea335f9d7555d33bf5cf25aa8c7d2504e3fa: Status 404 returned error can't find the container with id b0657f35534316731d0cae37705fea335f9d7555d33bf5cf25aa8c7d2504e3fa
Nov 04 00:57:52.674879 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:52.674842    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b" exitCode=0
Nov 04 00:57:52.675368 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:52.674927    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"}
Nov 04 00:57:52.675368 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:52.674990    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"b0657f35534316731d0cae37705fea335f9d7555d33bf5cf25aa8c7d2504e3fa"}
Nov 04 00:57:53.132421 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.132384    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:53.290624 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.290597    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:53.440313 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.440240    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:53.683257 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.683224    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerStarted","Data":"e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050"}
Nov 04 00:57:53.815025 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.814931    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:53.864001 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.863931    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.947202752 podStartE2EDuration="9.863912244s" totalImagesPullingTime="6.904019606s" totalInitContainerRuntime="12.689886ms" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:44 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:53.863243098 +0000 UTC m=+42.525383161" watchObservedRunningTime="2026-11-04 00:57:53.863912244 +0000 UTC m=+42.526052304"
Nov 04 00:57:53.992328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:53.992248    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:54.731990 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:54.731953    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:54.957369 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:54.957301    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:55.388210 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:55.388105    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:55.597825 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:55.597787    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:55.697292 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:55.697203    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d57f5b7d-cfbh4" event={"ID":"e70c70c2-2de4-4fc0-8019-87277af64a37","Type":"ContainerStarted","Data":"6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2"}
Nov 04 00:57:55.816557 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:55.816495    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-59d57f5b7d-cfbh4" podStartSLOduration=2.235338979 podStartE2EDuration="6.816479087s" totalImagesPullingTime="4.581140108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:55.816043849 +0000 UTC m=+44.478183912" watchObservedRunningTime="2026-11-04 00:57:55.816479087 +0000 UTC m=+44.478619148"
Nov 04 00:57:56.442909 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:56.442814    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:56.704574 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:56.704538    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"}
Nov 04 00:57:57.417088 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:57.417047    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:57:57.711104 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:57.711019    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"}
Nov 04 00:57:59.723797 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:59.723750    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-xhl8w" event={"ID":"ff8a5906-1d5b-4d35-bd99-3ed57cd09d90","Type":"ContainerStarted","Data":"1d66f72288631e6e2826858293c35715e56393c5f0e6d20c0cefb59ae318f984"}
Nov 04 00:57:59.724266 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:59.723953    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:59.743401 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:59.743373    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-xhl8w"
Nov 04 00:57:59.840589 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:59.840534    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-xhl8w" podStartSLOduration=2.779740436 podStartE2EDuration="22.840515003s" totalImagesPullingTime="20.060774567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:57:59.839288758 +0000 UTC m=+48.501428821" watchObservedRunningTime="2026-11-04 00:57:59.840515003 +0000 UTC m=+48.502655064"
Nov 04 00:57:59.850871 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:57:59.849291    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:00.048577 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.048543    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:00.160632 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.160601    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:58:00.160784 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.160734    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:58:00.168799 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.168772    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:58:00.733572 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.733523    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"}
Nov 04 00:58:00.734055 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.733616    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"}
Nov 04 00:58:00.734055 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.733659    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"}
Nov 04 00:58:00.734055 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.733713    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerStarted","Data":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"}
Nov 04 00:58:00.740409 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.740384    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:58:00.941093 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:00.941025    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.926831871 podStartE2EDuration="9.941004201s" totalImagesPullingTime="6.995969512s" totalInitContainerRuntime="18.202818ms" isStatefulPod=false podCreationTimestamp="2026-11-04 00:57:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:58:00.90787463 +0000 UTC m=+49.570014710" watchObservedRunningTime="2026-11-04 00:58:00.941004201 +0000 UTC m=+49.603144263"
Nov 04 00:58:01.933796 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:01.933753    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:58:03.047328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:03.047282    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:03.213062 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:03.213024    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:07.733577 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:07.733541    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:58:07.733990 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:07.733601    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:58:08.795901 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:08.795869    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:08.961492 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:08.961459    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:11.248997 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:11.248962    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:11.479335 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:11.479303    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:11.647309 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:11.647283    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:11.817265 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:11.817232    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:13.766587 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:13.766551    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59d57f5b7d-cfbh4"]
Nov 04 00:58:14.037025 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:14.036957    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:14.196806 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:14.196777    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:21.896113 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:58:21.896073    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 00:58:22.462083 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:22.462041    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:22.625076 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:22.625043    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:23.800965 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:23.800931    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:23.963955 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:23.963926    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:27.738509 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:27.738479    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:58:27.742097 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:27.742074    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-9958fc47b-vts5v"
Nov 04 00:58:28.321839 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:28.321800    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:28.703419 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:28.703373    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:38.784389 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:38.784355    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:38.786175 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:38.786149    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59d57f5b7d-cfbh4" podUID="e70c70c2-2de4-4fc0-8019-87277af64a37" containerName="console" containerID="cri-o://6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2" gracePeriod=15
Nov 04 00:58:38.945800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:38.945770    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:58:39.059123 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.059066    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:58:39.139481 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139451    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139610 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139499    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139655 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139630    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139735 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139661    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp\" (UniqueName: \"kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139735 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139724    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139820 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139768    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj\" (UniqueName: \"kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139820 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139807    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config\") pod \"e70c70c2-2de4-4fc0-8019-87277af64a37\" (UID: \"e70c70c2-2de4-4fc0-8019-87277af64a37\") "
Nov 04 00:58:39.139921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139865    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:58:39.139992 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.139968    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 00:58:39.140130 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.140113    4434 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-oauth-serving-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.140191 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.140134    4434 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e70c70c2-2de4-4fc0-8019-87277af64a37-tmp\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.140255 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.140209    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:58:39.140301 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.140274    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:58:39.141709 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.141684    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:58:39.141796 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.141769    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "kube-api-access-8zcsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 00:58:39.141850 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.141829    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert" pod "e70c70c2-2de4-4fc0-8019-87277af64a37" (UID: "e70c70c2-2de4-4fc0-8019-87277af64a37"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:58:39.240937 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.240905    4434 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-oauth-config\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.240937 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.240931    4434 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e70c70c2-2de4-4fc0-8019-87277af64a37-console-serving-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.240937 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.240941    4434 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.241152 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.240950    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zcsj\" (UniqueName: \"kubernetes.io/projected/e70c70c2-2de4-4fc0-8019-87277af64a37-kube-api-access-8zcsj\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.241152 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.240959    4434 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e70c70c2-2de4-4fc0-8019-87277af64a37-console-config\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:58:39.872904 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.872831    4434 generic.go:356] "Generic (PLEG): container finished" podID="e70c70c2-2de4-4fc0-8019-87277af64a37" containerID="6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2" exitCode=0
Nov 04 00:58:39.872904 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.872879    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d57f5b7d-cfbh4"
Nov 04 00:58:39.873328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.872918    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d57f5b7d-cfbh4" event={"ID":"e70c70c2-2de4-4fc0-8019-87277af64a37","Type":"ContainerDied","Data":"6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2"}
Nov 04 00:58:39.873328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.872970    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d57f5b7d-cfbh4" event={"ID":"e70c70c2-2de4-4fc0-8019-87277af64a37","Type":"ContainerDied","Data":"40f59f86049efe88c04787dc0de043bf4d031d33dcf16b8e385b9b80b380aba2"}
Nov 04 00:58:39.873328 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.872996    4434 scope.go:122] "RemoveContainer" containerID="6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2"
Nov 04 00:58:39.880959 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.880937    4434 scope.go:122] "RemoveContainer" containerID="6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2"
Nov 04 00:58:39.881218 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:58:39.881198    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2\": container with ID starting with 6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2 not found: ID does not exist" containerID="6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2"
Nov 04 00:58:39.881271 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:39.881228    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2"} err="failed to get container status \"6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2\": rpc error: code = NotFound desc = could not find container \"6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2\": container with ID starting with 6995f6787f6e12d78202bb7ea10daf467622cb64440acd99f9773757d2b9b5b2 not found: ID does not exist"
Nov 04 00:58:40.037772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:40.037743    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59d57f5b7d-cfbh4"]
Nov 04 00:58:40.088181 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:40.088147    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59d57f5b7d-cfbh4"]
Nov 04 00:58:41.471975 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:41.471945    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e70c70c2-2de4-4fc0-8019-87277af64a37" path="/var/lib/kubelet/pods/e70c70c2-2de4-4fc0-8019-87277af64a37/volumes"
Nov 04 00:58:51.896754 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:58:51.896718    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 00:58:53.236488 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:53.236450    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:58:53.254930 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:53.254905    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:58:53.945040 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:53.945012    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:58:55.193680 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:58:55.193631    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:06.349876 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:06.349840    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:06.559474 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:06.559443    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:08.590445 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.590415    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:59:08.590858 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.590756    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="alertmanager" containerID="cri-o://e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c" gracePeriod=120
Nov 04 00:59:08.590858 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.590769    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="config-reloader" containerID="cri-o://bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f" gracePeriod=120
Nov 04 00:59:08.590858 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.590772    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-metric" containerID="cri-o://12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511" gracePeriod=120
Nov 04 00:59:08.591036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.590851    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy" containerID="cri-o://b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e" gracePeriod=120
Nov 04 00:59:08.591036 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.590899    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="prom-label-proxy" containerID="cri-o://e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050" gracePeriod=120
Nov 04 00:59:08.591135 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.591025    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-web" containerID="cri-o://35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c" gracePeriod=120
Nov 04 00:59:08.983912 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.983882    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050" exitCode=0
Nov 04 00:59:08.983912 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.983907    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511" exitCode=0
Nov 04 00:59:08.983912 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.983913    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e" exitCode=0
Nov 04 00:59:08.983912 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.983920    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f" exitCode=0
Nov 04 00:59:08.984180 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.983926    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c" exitCode=0
Nov 04 00:59:08.984180 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.983958    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050"}
Nov 04 00:59:08.984180 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.984036    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511"}
Nov 04 00:59:08.984180 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.984065    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e"}
Nov 04 00:59:08.984180 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.984089    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f"}
Nov 04 00:59:08.984180 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:08.984113    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c"}
Nov 04 00:59:09.841656 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.841634    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:09.897427 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897400    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897534 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897435    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897534 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897467    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897534 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897487    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897534 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897505    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897553    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897584    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897608    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897643    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897694    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.897772 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.897738    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.899080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.898833    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 00:59:09.899080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.899034    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:09.900264 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.900234    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 00:59:09.900605 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.900563    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "kube-api-access-x62vj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 00:59:09.900984 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.900812    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:09.900984 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.900899    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:09.901134 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.901020    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:09.901178 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.901128    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 00:59:09.901178 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.901165    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:09.904317 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.904215    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:09.910532 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.910507    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:09.960369 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.960312    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:09.990556 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.990530    4434 generic.go:356] "Generic (PLEG): container finished" podID="a034a8b0-8581-4186-84d8-573a5f049e67" containerID="35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c" exitCode=0
Nov 04 00:59:09.990650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.990617    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c"}
Nov 04 00:59:09.990650 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.990638    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:09.990738 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.990708    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a034a8b0-8581-4186-84d8-573a5f049e67","Type":"ContainerDied","Data":"a76b64394b0210fc5fcbe19c8a5eecd7b94d2dc7c260e2808ac3fe9019946f21"}
Nov 04 00:59:09.990770 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.990742    4434 scope.go:122] "RemoveContainer" containerID="e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050"
Nov 04 00:59:09.997605 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.997591    4434 scope.go:122] "RemoveContainer" containerID="12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511"
Nov 04 00:59:09.998182 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998165    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.998269 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998193    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web\") pod \"a034a8b0-8581-4186-84d8-573a5f049e67\" (UID: \"a034a8b0-8581-4186-84d8-573a5f049e67\") "
Nov 04 00:59:09.998407 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998362    4434 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-web-config\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998407 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998379    4434 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-main-db\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998407 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998388    4434 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-cluster-tls-config\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998407 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998398    4434 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998407 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998407    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-metrics-client-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998416    4434 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998424    4434 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a034a8b0-8581-4186-84d8-573a5f049e67-config-out\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998433    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x62vj\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-kube-api-access-x62vj\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998446    4434 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-main-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998457    4434 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-config-volume\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998465    4434 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a034a8b0-8581-4186-84d8-573a5f049e67-tls-assets\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:09.998948 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.998879    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:09.999999 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:09.999970    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web" pod "a034a8b0-8581-4186-84d8-573a5f049e67" (UID: "a034a8b0-8581-4186-84d8-573a5f049e67"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:10.004162 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.004145    4434 scope.go:122] "RemoveContainer" containerID="b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e"
Nov 04 00:59:10.010467 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.010451    4434 scope.go:122] "RemoveContainer" containerID="35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c"
Nov 04 00:59:10.016651 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.016634    4434 scope.go:122] "RemoveContainer" containerID="bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f"
Nov 04 00:59:10.023003 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.022986    4434 scope.go:122] "RemoveContainer" containerID="e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c"
Nov 04 00:59:10.029316 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.029299    4434 scope.go:122] "RemoveContainer" containerID="06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e"
Nov 04 00:59:10.035510 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.035492    4434 scope.go:122] "RemoveContainer" containerID="e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050"
Nov 04 00:59:10.035774 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.035755    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050\": container with ID starting with e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050 not found: ID does not exist" containerID="e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050"
Nov 04 00:59:10.035822 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.035783    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050"} err="failed to get container status \"e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050\": rpc error: code = NotFound desc = could not find container \"e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050\": container with ID starting with e32a751afd9f5f44cc3d99202b7f179bed1d17719d7b9324541ecd7d7bd37050 not found: ID does not exist"
Nov 04 00:59:10.035822 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.035801    4434 scope.go:122] "RemoveContainer" containerID="12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511"
Nov 04 00:59:10.036042 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.036021    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511\": container with ID starting with 12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511 not found: ID does not exist" containerID="12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511"
Nov 04 00:59:10.036078 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036048    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511"} err="failed to get container status \"12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511\": rpc error: code = NotFound desc = could not find container \"12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511\": container with ID starting with 12d34b3a6dfef06cdc98aa9304b01da29360bc0f6c1229f25a6ed28029ea8511 not found: ID does not exist"
Nov 04 00:59:10.036078 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036064    4434 scope.go:122] "RemoveContainer" containerID="b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e"
Nov 04 00:59:10.036281 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.036264    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e\": container with ID starting with b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e not found: ID does not exist" containerID="b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e"
Nov 04 00:59:10.036317 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036287    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e"} err="failed to get container status \"b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e\": rpc error: code = NotFound desc = could not find container \"b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e\": container with ID starting with b64f0e8a74ee2e2208ce44a3c911edf3c88d73762176cbeed52beb01183d395e not found: ID does not exist"
Nov 04 00:59:10.036317 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036300    4434 scope.go:122] "RemoveContainer" containerID="35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c"
Nov 04 00:59:10.036521 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.036504    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c\": container with ID starting with 35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c not found: ID does not exist" containerID="35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c"
Nov 04 00:59:10.036561 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036529    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c"} err="failed to get container status \"35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c\": rpc error: code = NotFound desc = could not find container \"35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c\": container with ID starting with 35c3b19ceffc9f3370a03136ebd6217fd45a33074bc48d1da14e3e28ab89de5c not found: ID does not exist"
Nov 04 00:59:10.036561 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036544    4434 scope.go:122] "RemoveContainer" containerID="bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f"
Nov 04 00:59:10.036770 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.036754    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f\": container with ID starting with bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f not found: ID does not exist" containerID="bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f"
Nov 04 00:59:10.036820 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036774    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f"} err="failed to get container status \"bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f\": rpc error: code = NotFound desc = could not find container \"bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f\": container with ID starting with bc264a327151a1a12b34da731a42ecb142beb15bd032f7428157265c5e809d2f not found: ID does not exist"
Nov 04 00:59:10.036820 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.036785    4434 scope.go:122] "RemoveContainer" containerID="e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c"
Nov 04 00:59:10.037003 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.036982    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c\": container with ID starting with e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c not found: ID does not exist" containerID="e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c"
Nov 04 00:59:10.037048 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.037008    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c"} err="failed to get container status \"e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c\": rpc error: code = NotFound desc = could not find container \"e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c\": container with ID starting with e88da6da127a9f43acb32856eef6c6e2e00e2bae964c52b4bf2cb5662106890c not found: ID does not exist"
Nov 04 00:59:10.037048 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.037020    4434 scope.go:122] "RemoveContainer" containerID="06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e"
Nov 04 00:59:10.037213 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:10.037198    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e\": container with ID starting with 06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e not found: ID does not exist" containerID="06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e"
Nov 04 00:59:10.037255 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.037217    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e"} err="failed to get container status \"06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e\": rpc error: code = NotFound desc = could not find container \"06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e\": container with ID starting with 06dd23047bf056190aac8bae381a6f38ec5ef423bb2cb68b01d475f36671066e not found: ID does not exist"
Nov 04 00:59:10.099623 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.099594    4434 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a034a8b0-8581-4186-84d8-573a5f049e67-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:10.099623 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.099617    4434 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a034a8b0-8581-4186-84d8-573a5f049e67-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:10.161746 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.161714    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:10.414681 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.414647    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:59:10.419997 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.419970    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:59:10.468078 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468053    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:59:10.468342 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468328    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-metric"
Nov 04 00:59:10.468384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468345    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-metric"
Nov 04 00:59:10.468384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468366    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="init-config-reloader"
Nov 04 00:59:10.468384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468371    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="init-config-reloader"
Nov 04 00:59:10.468384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468380    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-web"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468386    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-web"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468398    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="config-reloader"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468402    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="config-reloader"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468411    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468416    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468429    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e70c70c2-2de4-4fc0-8019-87277af64a37" containerName="console"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468437    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e70c70c2-2de4-4fc0-8019-87277af64a37" containerName="console"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468446    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="alertmanager"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468450    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="alertmanager"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468460    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="prom-label-proxy"
Nov 04 00:59:10.468500 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468464    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="prom-label-proxy"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468504    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e70c70c2-2de4-4fc0-8019-87277af64a37" containerName="console"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468515    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="alertmanager"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468522    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="prom-label-proxy"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468528    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-metric"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468535    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy-web"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468541    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="config-reloader"
Nov 04 00:59:10.468817 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.468547    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" containerName="kube-rbac-proxy"
Nov 04 00:59:10.472872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.472854    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.503589 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503557    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38b8e65b-f464-4e11-9db7-756974868b61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.503710 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503592    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38b8e65b-f464-4e11-9db7-756974868b61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.503710 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503624    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-web-config\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.503805 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503783    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38b8e65b-f464-4e11-9db7-756974868b61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.503860 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503817    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38b8e65b-f464-4e11-9db7-756974868b61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.503940 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503923    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38b8e65b-f464-4e11-9db7-756974868b61-config-out\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504011 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503951    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504011 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.503993    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdj5b\" (UniqueName: \"kubernetes.io/projected/38b8e65b-f464-4e11-9db7-756974868b61-kube-api-access-rdj5b\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504126 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.504027    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504126 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.504100    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504221 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.504133    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504332 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.504292    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.504457 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.504350    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-config-volume\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.527045 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527021    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 04 00:59:10.527045 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527041    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 04 00:59:10.527204 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527055    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 04 00:59:10.527204 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527089    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 04 00:59:10.527204 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527115    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 04 00:59:10.527204 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527124    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 04 00:59:10.527374 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.527205    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 04 00:59:10.576536 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.576476    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 04 00:59:10.576651 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.576634    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-phxps\""
Nov 04 00:59:10.576745 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.576728    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 04 00:59:10.577856 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.577838    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:59:10.605079 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605051    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-config-volume\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605213 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605086    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38b8e65b-f464-4e11-9db7-756974868b61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605213 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605115    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38b8e65b-f464-4e11-9db7-756974868b61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605213 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605203    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-web-config\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605317 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605245    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38b8e65b-f464-4e11-9db7-756974868b61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605317 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605267    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38b8e65b-f464-4e11-9db7-756974868b61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605317 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605303    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38b8e65b-f464-4e11-9db7-756974868b61-config-out\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605461 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605328    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605549 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605527    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/38b8e65b-f464-4e11-9db7-756974868b61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605579    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdj5b\" (UniqueName: \"kubernetes.io/projected/38b8e65b-f464-4e11-9db7-756974868b61-kube-api-access-rdj5b\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605625 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605617    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605766 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605651    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605766 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605702    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605766 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605743    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.605979 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.605957    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38b8e65b-f464-4e11-9db7-756974868b61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.606142 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.606108    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38b8e65b-f464-4e11-9db7-756974868b61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.608406 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.608377    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-config-volume\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.608540 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.608519    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.608593 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.608519    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.608780 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.608760    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38b8e65b-f464-4e11-9db7-756974868b61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.608872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.608763    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38b8e65b-f464-4e11-9db7-756974868b61-config-out\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.609068 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.609048    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.609120 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.609048    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-web-config\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.609247 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.609223    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.610232 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.610213    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/38b8e65b-f464-4e11-9db7-756974868b61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.663000 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.662976    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdj5b\" (UniqueName: \"kubernetes.io/projected/38b8e65b-f464-4e11-9db7-756974868b61-kube-api-access-rdj5b\") pod \"alertmanager-main-0\" (UID: \"38b8e65b-f464-4e11-9db7-756974868b61\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:10.788814 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:10.788779    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 04 00:59:11.216818 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:11.216782    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 04 00:59:11.220688 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:59:11.220638    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b8e65b_f464_4e11_9db7_756974868b61.slice/crio-1339570fa7aa5ad2beb8b12094e0d994ed2bd2ccbf472e811e576d980a1ef293 WatchSource:0}: Error finding container 1339570fa7aa5ad2beb8b12094e0d994ed2bd2ccbf472e811e576d980a1ef293: Status 404 returned error can't find the container with id 1339570fa7aa5ad2beb8b12094e0d994ed2bd2ccbf472e811e576d980a1ef293
Nov 04 00:59:11.471885 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:11.471835    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a034a8b0-8581-4186-84d8-573a5f049e67" path="/var/lib/kubelet/pods/a034a8b0-8581-4186-84d8-573a5f049e67/volumes"
Nov 04 00:59:11.999543 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:11.999511    4434 generic.go:356] "Generic (PLEG): container finished" podID="38b8e65b-f464-4e11-9db7-756974868b61" containerID="4d8463c8a67db558f7d14f5876d382c71a7987a6a4d665b9e4693a2da57c00ec" exitCode=0
Nov 04 00:59:11.999718 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:11.999601    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerDied","Data":"4d8463c8a67db558f7d14f5876d382c71a7987a6a4d665b9e4693a2da57c00ec"}
Nov 04 00:59:11.999718 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:11.999706    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"1339570fa7aa5ad2beb8b12094e0d994ed2bd2ccbf472e811e576d980a1ef293"}
Nov 04 00:59:12.196812 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.196787    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-797775ffd6-zlbr7"]
Nov 04 00:59:12.201861 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.201844    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221548 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221514    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221572    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221617    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-secret-telemeter-client\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221654    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-federate-client-tls\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221717    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-telemeter-client-tls\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221744    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-serving-certs-ca-bundle\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221761    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-metrics-client-ca\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.221921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.221777    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6djz\" (UniqueName: \"kubernetes.io/projected/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-kube-api-access-n6djz\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.254773 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.254751    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Nov 04 00:59:12.254908 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.254876    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Nov 04 00:59:12.255096 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.254980    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Nov 04 00:59:12.255096 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.255009    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Nov 04 00:59:12.255096 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.255073    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Nov 04 00:59:12.255238 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.255128    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2jtpf\""
Nov 04 00:59:12.311227 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.307841    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Nov 04 00:59:12.311227 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.308071    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-797775ffd6-zlbr7"]
Nov 04 00:59:12.322969 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.322946    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-serving-certs-ca-bundle\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323040 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.322981    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-metrics-client-ca\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323040 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323002    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6djz\" (UniqueName: \"kubernetes.io/projected/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-kube-api-access-n6djz\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323120 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323047    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323120 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323080    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323252 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323234    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-secret-telemeter-client\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323653 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323307    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-federate-client-tls\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323653 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323378    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-telemeter-client-tls\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323818 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323642    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-serving-certs-ca-bundle\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.323862 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.323810    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-metrics-client-ca\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.324109 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.324092    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-telemeter-trusted-ca-bundle\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.326161 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.326124    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-federate-client-tls\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.326276 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.326160    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-secret-telemeter-client\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.326276 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.326177    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-telemeter-client-tls\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.326276 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.326223    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.379351 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.379327    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6djz\" (UniqueName: \"kubernetes.io/projected/5f4cc4c0-41fa-45cd-ae33-60570b10b12e-kube-api-access-n6djz\") pod \"telemeter-client-797775ffd6-zlbr7\" (UID: \"5f4cc4c0-41fa-45cd-ae33-60570b10b12e\") " pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.513966 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.513900    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7"
Nov 04 00:59:12.936281 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:12.936248    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-797775ffd6-zlbr7"]
Nov 04 00:59:12.940227 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:59:12.940188    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4cc4c0_41fa_45cd_ae33_60570b10b12e.slice/crio-8126df3c32e4630ab56eb19248cfc4ee59d3c9ed0430f11e90b19b721071e044 WatchSource:0}: Error finding container 8126df3c32e4630ab56eb19248cfc4ee59d3c9ed0430f11e90b19b721071e044: Status 404 returned error can't find the container with id 8126df3c32e4630ab56eb19248cfc4ee59d3c9ed0430f11e90b19b721071e044
Nov 04 00:59:13.004482 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.004451    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7" event={"ID":"5f4cc4c0-41fa-45cd-ae33-60570b10b12e","Type":"ContainerStarted","Data":"8126df3c32e4630ab56eb19248cfc4ee59d3c9ed0430f11e90b19b721071e044"}
Nov 04 00:59:13.007510 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.007483    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"1ed27890c6feaa603ef9cfd0b7e1925b3bb7be3e2cff165fe1cdd0e95f11c196"}
Nov 04 00:59:13.007596 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.007538    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"67bdf86697b856f167e828d90bc2ec190d5ef1529f468367d60c39429fa716b7"}
Nov 04 00:59:13.007596 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.007562    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"16869e30a64fc6950427f38f7d28101c477fddc8c51f116c72f651e5b841d474"}
Nov 04 00:59:13.007596 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.007592    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"186c4ea04124b8dd3c8cae054e4c46e331d2cd363d586ccdf2b07652fa0cea2d"}
Nov 04 00:59:13.007737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.007618    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"eccbb9df497bac72bf6d87843f50dfca1e87dc36af1b7ca678a3797d292443cb"}
Nov 04 00:59:13.007737 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.007643    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"38b8e65b-f464-4e11-9db7-756974868b61","Type":"ContainerStarted","Data":"a32cfce90255ca90948781490ba09870c6e95ca1c95f86b5130267f7e931b633"}
Nov 04 00:59:13.135466 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:13.135419    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.124351118 podStartE2EDuration="3.135405822s" totalImagesPullingTime="0s" totalInitContainerRuntime="11.054704ms" isStatefulPod=false podCreationTimestamp="2026-11-04 00:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:59:13.134828565 +0000 UTC m=+121.796968650" watchObservedRunningTime="2026-11-04 00:59:13.135405822 +0000 UTC m=+121.797545877"
Nov 04 00:59:14.229470 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229435    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:59:14.229915 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229724    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="prometheus" containerID="cri-o://4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756" gracePeriod=600
Nov 04 00:59:14.229915 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229762    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-thanos" containerID="cri-o://dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f" gracePeriod=600
Nov 04 00:59:14.229915 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229795    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-web" containerID="cri-o://fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1" gracePeriod=600
Nov 04 00:59:14.229915 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229805    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy" containerID="cri-o://e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6" gracePeriod=600
Nov 04 00:59:14.229915 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229810    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="thanos-sidecar" containerID="cri-o://00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7" gracePeriod=600
Nov 04 00:59:14.229915 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.229824    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="config-reloader" containerID="cri-o://54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e" gracePeriod=600
Nov 04 00:59:14.482603 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.482549    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:14.540456 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540433    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540569 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540461    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540569 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540485    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540569 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540503    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540569 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540535    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540818 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540586    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540818 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540634    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540925 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540876    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.540925 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540893    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.540925 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540914    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.541080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540932    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.540962    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541000    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541023    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.541080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541033    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541080 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541067    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541098    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541134    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541162    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541194    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541213    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541230    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541263    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541384 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541303    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets\") pod \"aafe79fb-4c72-42de-b4df-a8127425187a\" (UID: \"aafe79fb-4c72-42de-b4df-a8127425187a\") "
Nov 04 00:59:14.541803 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541570    4434 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-metrics-client-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.541803 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541590    4434 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.541803 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541610    4434 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.541803 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541626    4434 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.542012 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.541960    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 00:59:14.542063 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.542005    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.542458 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.542425    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.543142 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.543113    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.543486 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.543461    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.544697 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.544645    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 00:59:14.545229 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.545202    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.545782 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.545390    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.545782 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.545511    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.545782 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.545578    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "kube-api-access-4fgxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 00:59:14.545782 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.545747    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 00:59:14.546010 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.545858    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.546302 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.546283    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 00:59:14.546360 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.546289    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.546695 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.546653    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.554274 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.554255    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config" pod "aafe79fb-4c72-42de-b4df-a8127425187a" (UID: "aafe79fb-4c72-42de-b4df-a8127425187a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 00:59:14.642436 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642407    4434 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-web-config\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642436 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642434    4434 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642444    4434 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642455    4434 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-db\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642464    4434 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-kube-rbac-proxy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642472    4434 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-config\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642486    4434 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-metrics-client-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642498    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fgxj\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-kube-api-access-4fgxj\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642507    4434 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-grpc-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642515    4434 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aafe79fb-4c72-42de-b4df-a8127425187a-config-out\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642524    4434 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642534    4434 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642543    4434 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642552    4434 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/aafe79fb-4c72-42de-b4df-a8127425187a-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642560    4434 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aafe79fb-4c72-42de-b4df-a8127425187a-tls-assets\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:14.642621 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:14.642568    4434 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/aafe79fb-4c72-42de-b4df-a8127425187a-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 00:59:15.020592 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020561    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f" exitCode=0
Nov 04 00:59:15.020592 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020588    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6" exitCode=0
Nov 04 00:59:15.020592 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020594    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1" exitCode=0
Nov 04 00:59:15.020592 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020602    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7" exitCode=0
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020608    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e" exitCode=0
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020613    4434 generic.go:356] "Generic (PLEG): container finished" podID="aafe79fb-4c72-42de-b4df-a8127425187a" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756" exitCode=0
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020643    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020650    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"}
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020736    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"}
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020765    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"}
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020779    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020789    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"}
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020816    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"}
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020840    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"}
Nov 04 00:59:15.020920 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.020863    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"aafe79fb-4c72-42de-b4df-a8127425187a","Type":"ContainerDied","Data":"b0657f35534316731d0cae37705fea335f9d7555d33bf5cf25aa8c7d2504e3fa"}
Nov 04 00:59:15.028809 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.028446    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.035346 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.035331    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.041593 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.041566    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.047831 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.047815    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.054312 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.054294    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.060948 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.060929    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.067028 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067011    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.067288 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.067269    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.067332 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067300    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"} err="failed to get container status \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist"
Nov 04 00:59:15.067332 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067317    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.067541 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.067526    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.067584 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067544    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"} err="failed to get container status \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist"
Nov 04 00:59:15.067584 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067557    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.067860 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.067843    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.067921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067861    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"} err="failed to get container status \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist"
Nov 04 00:59:15.067921 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.067873    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.068117 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.068098    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.068166 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068123    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"} err="failed to get container status \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist"
Nov 04 00:59:15.068166 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068143    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.068361 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.068346    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.068409 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068364    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"} err="failed to get container status \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist"
Nov 04 00:59:15.068409 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068376    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.068614 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.068597    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.068681 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068619    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"} err="failed to get container status \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist"
Nov 04 00:59:15.068681 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068641    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.068887 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:15.068871    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.068922 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068890    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"} err="failed to get container status \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist"
Nov 04 00:59:15.068922 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.068901    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.069123 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069108    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"} err="failed to get container status \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist"
Nov 04 00:59:15.069123 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069123    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.069314 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069295    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"} err="failed to get container status \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist"
Nov 04 00:59:15.069362 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069313    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.069511 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069490    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"} err="failed to get container status \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist"
Nov 04 00:59:15.069551 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069511    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.069733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069718    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"} err="failed to get container status \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist"
Nov 04 00:59:15.069733 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069732    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.069919 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069905    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"} err="failed to get container status \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist"
Nov 04 00:59:15.069919 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.069918    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.070097 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070083    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"} err="failed to get container status \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist"
Nov 04 00:59:15.070097 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070096    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.070273 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070257    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"} err="failed to get container status \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist"
Nov 04 00:59:15.070313 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070273    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.070471 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070439    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"} err="failed to get container status \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist"
Nov 04 00:59:15.070508 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070472    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.070647 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070631    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"} err="failed to get container status \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist"
Nov 04 00:59:15.070699 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070647    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.070916 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070898    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"} err="failed to get container status \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist"
Nov 04 00:59:15.070916 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.070914    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.071105 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071090    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"} err="failed to get container status \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist"
Nov 04 00:59:15.071105 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071103    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.071281 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071267    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"} err="failed to get container status \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist"
Nov 04 00:59:15.071281 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071279    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.071459 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071443    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"} err="failed to get container status \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist"
Nov 04 00:59:15.071459 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071456    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.071633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071618    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"} err="failed to get container status \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist"
Nov 04 00:59:15.071633 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071630    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.071827 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071808    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"} err="failed to get container status \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist"
Nov 04 00:59:15.071872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071827    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.072011 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.071996    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"} err="failed to get container status \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist"
Nov 04 00:59:15.072053 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072011    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.072192 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072177    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"} err="failed to get container status \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist"
Nov 04 00:59:15.072228 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072191    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.072367 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072352    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"} err="failed to get container status \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist"
Nov 04 00:59:15.072407 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072367    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.072544 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072529    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"} err="failed to get container status \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist"
Nov 04 00:59:15.072590 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072544    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.072739 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072724    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"} err="failed to get container status \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist"
Nov 04 00:59:15.072778 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072739    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.072940 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072923    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"} err="failed to get container status \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist"
Nov 04 00:59:15.072940 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.072938    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.073121 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073106    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"} err="failed to get container status \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist"
Nov 04 00:59:15.073162 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073121    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.073301 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073286    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"} err="failed to get container status \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist"
Nov 04 00:59:15.073339 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073301    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.073478 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073462    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"} err="failed to get container status \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist"
Nov 04 00:59:15.073478 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073477    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.073700 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073677    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"} err="failed to get container status \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist"
Nov 04 00:59:15.073766 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073753    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.073993 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073976    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"} err="failed to get container status \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist"
Nov 04 00:59:15.074056 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.073993    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.074205 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074190    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"} err="failed to get container status \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist"
Nov 04 00:59:15.074244 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074205    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.074394 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074379    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"} err="failed to get container status \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist"
Nov 04 00:59:15.074437 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074395    4434 scope.go:122] "RemoveContainer" containerID="dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"
Nov 04 00:59:15.074613 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074596    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f"} err="failed to get container status \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": rpc error: code = NotFound desc = could not find container \"dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f\": container with ID starting with dd8dfa8b55919830330b49135d5b2e8f92cd26ab6572b5b5c7f0009d6a5cc90f not found: ID does not exist"
Nov 04 00:59:15.074697 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074612    4434 scope.go:122] "RemoveContainer" containerID="e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"
Nov 04 00:59:15.074872 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074853    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6"} err="failed to get container status \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": rpc error: code = NotFound desc = could not find container \"e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6\": container with ID starting with e5984d5d623f7602fad617c6afde47e145c5ff1944ed21727eecec7ee75543a6 not found: ID does not exist"
Nov 04 00:59:15.074929 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.074873    4434 scope.go:122] "RemoveContainer" containerID="fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"
Nov 04 00:59:15.075111 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075094    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1"} err="failed to get container status \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": rpc error: code = NotFound desc = could not find container \"fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1\": container with ID starting with fa4c4fb6d5afe2c6e0c67b3e2dde7b849b4fef16d1a3a9ec66356d3b8a87fce1 not found: ID does not exist"
Nov 04 00:59:15.075156 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075111    4434 scope.go:122] "RemoveContainer" containerID="00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"
Nov 04 00:59:15.075335 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075317    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7"} err="failed to get container status \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": rpc error: code = NotFound desc = could not find container \"00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7\": container with ID starting with 00fa13ab0018f87aba8398737499d0392f292da8ac047b0bfb667da555cc69d7 not found: ID does not exist"
Nov 04 00:59:15.075393 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075335    4434 scope.go:122] "RemoveContainer" containerID="54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"
Nov 04 00:59:15.075546 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075529    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e"} err="failed to get container status \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": rpc error: code = NotFound desc = could not find container \"54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e\": container with ID starting with 54950340d25393d0512622369dd3530b2b5774810999d4ff70a59667e18e4f9e not found: ID does not exist"
Nov 04 00:59:15.075587 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075546    4434 scope.go:122] "RemoveContainer" containerID="4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"
Nov 04 00:59:15.075823 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075805    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756"} err="failed to get container status \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": rpc error: code = NotFound desc = could not find container \"4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756\": container with ID starting with 4a2727f1f3134ff8eb0dc524299d8434fb2b36dc9e6cbba25ad97b5b21553756 not found: ID does not exist"
Nov 04 00:59:15.075823 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.075822    4434 scope.go:122] "RemoveContainer" containerID="3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"
Nov 04 00:59:15.076030 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.076003    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b"} err="failed to get container status \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": rpc error: code = NotFound desc = could not find container \"3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b\": container with ID starting with 3b6f21b9f7b3ad658f5c2328f46fd57c4c99778b985c6fb43ab00144a3fde05b not found: ID does not exist"
Nov 04 00:59:15.191605 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.189727    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:59:15.193019 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.192987    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:59:15.238225 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238203    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238524    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="init-config-reloader"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238536    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="init-config-reloader"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238550    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="thanos-sidecar"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238556    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="thanos-sidecar"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238572    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="prometheus"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238577    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="prometheus"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238585    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="config-reloader"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238590    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="config-reloader"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238599    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-web"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238604    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-web"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238617    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238624    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238636    4434 cpu_manager.go:414] "containerMap: removing container" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-thanos"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238643    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-thanos"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238710    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-thanos"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238724    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="thanos-sidecar"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238735    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="prometheus"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238742    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="config-reloader"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238748    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy-web"
Nov 04 00:59:15.239114 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.238754    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" containerName="kube-rbac-proxy"
Nov 04 00:59:15.243054 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.243037    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.297086 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297024    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 04 00:59:15.297086 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297057    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 04 00:59:15.297198 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297174    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 04 00:59:15.297198 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297181    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 04 00:59:15.297282 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297225    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 04 00:59:15.297282 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297232    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-xpqx4\""
Nov 04 00:59:15.297282 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297230    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-3gjddat11co3b\""
Nov 04 00:59:15.297391 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297245    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 04 00:59:15.297391 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297299    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 04 00:59:15.297391 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297288    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 04 00:59:15.297391 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.297286    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 04 00:59:15.339064 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.339040    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 04 00:59:15.339064 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.339041    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 04 00:59:15.339307 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.339138    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 04 00:59:15.349268 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349241    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349361 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349277    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6ncr\" (UniqueName: \"kubernetes.io/projected/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-kube-api-access-z6ncr\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349361 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349300    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-config\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349361 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349321    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349481 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349431    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349481 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349449    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349481 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349466    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349580 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349484    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349619 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349581    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349658 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349618    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349658 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349650    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349792 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349744    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349792 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349780    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349908 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349888    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.349964 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349927    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.350000 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349959    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.350000 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.349991    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.350061 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.350016    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.350061 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.350050    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.350196 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.350076    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.390040 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.390017    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 04 00:59:15.390155 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.390130    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 04 00:59:15.391360 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.391344    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:59:15.450727 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450699    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.450844 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450735    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.450844 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450759    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.450844 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450784    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.450844 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450803    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.450844 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450834    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450859    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450882    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450906    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6ncr\" (UniqueName: \"kubernetes.io/projected/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-kube-api-access-z6ncr\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450936    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-config\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450952    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.450989    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451004    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451021    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451036    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451059    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451092 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451078    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451630 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451098    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451630 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451127    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451630 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451152    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.451630 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.451258    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.452293 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.452077    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.452293 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.452080    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.452434 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.452296    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.453481 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.452584    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.453597 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.453554    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.453905 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.453875    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.454837 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.454775    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.454925 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.454878    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.456347 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.456293    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-config-out\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.456612 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.456583    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-web-config\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.456612 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.456588    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.457398 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.457375    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-config\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.457479 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.457380    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.458169 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.457771    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.458169 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.458132    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.458169 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.458135    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.459442 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.459419    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.460066 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.460046    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.475855 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.475829    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aafe79fb-4c72-42de-b4df-a8127425187a" path="/var/lib/kubelet/pods/aafe79fb-4c72-42de-b4df-a8127425187a/volumes"
Nov 04 00:59:15.509755 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.509735    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6ncr\" (UniqueName: \"kubernetes.io/projected/7e75b897-7a6b-45d1-bfbd-2bc050c490d1-kube-api-access-z6ncr\") pod \"prometheus-k8s-0\" (UID: \"7e75b897-7a6b-45d1-bfbd-2bc050c490d1\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:15.574408 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:15.574345    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:16.055754 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:16.055718    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 04 00:59:16.059617 ip-10-0-136-128 kubenswrapper[4434]: W1104 00:59:16.059588    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e75b897_7a6b_45d1_bfbd_2bc050c490d1.slice/crio-352ec0b631e804a542c235b3839787513c243920710ad1f3f2ef07a002c63458 WatchSource:0}: Error finding container 352ec0b631e804a542c235b3839787513c243920710ad1f3f2ef07a002c63458: Status 404 returned error can't find the container with id 352ec0b631e804a542c235b3839787513c243920710ad1f3f2ef07a002c63458
Nov 04 00:59:17.030805 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:17.030771    4434 generic.go:356] "Generic (PLEG): container finished" podID="7e75b897-7a6b-45d1-bfbd-2bc050c490d1" containerID="99678e7eec7797abe4690a9e13687d3412ecd5163183989d9a718072a104997d" exitCode=0
Nov 04 00:59:17.031277 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:17.030855    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerDied","Data":"99678e7eec7797abe4690a9e13687d3412ecd5163183989d9a718072a104997d"}
Nov 04 00:59:17.031277 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:17.030929    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"352ec0b631e804a542c235b3839787513c243920710ad1f3f2ef07a002c63458"}
Nov 04 00:59:18.039405 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.039357    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7" event={"ID":"5f4cc4c0-41fa-45cd-ae33-60570b10b12e","Type":"ContainerStarted","Data":"91d49973a62b36c9ffddd2bbae398e05f6320b8bb03249c80a4ccd8633bf7066"}
Nov 04 00:59:18.039854 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.039431    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7" event={"ID":"5f4cc4c0-41fa-45cd-ae33-60570b10b12e","Type":"ContainerStarted","Data":"068b24ac5596987b95f77dbd386b837895ebfe2750079be83eef3be4c444bc9c"}
Nov 04 00:59:18.039854 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.039462    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7" event={"ID":"5f4cc4c0-41fa-45cd-ae33-60570b10b12e","Type":"ContainerStarted","Data":"b1d540021c1d372bb9e41cfc11952ee151776e709cf983c248a35996e9c1055f"}
Nov 04 00:59:18.042506 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.042476    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"b662dac1043eff7fb1c8f6bd29563ed286ebea42be4ce37395696d991631c925"}
Nov 04 00:59:18.042629 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.042534    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"a50ae534e51174aa6d90d9f3fff918c928e2fda6b9cdc87f9222bbc5ba6d2652"}
Nov 04 00:59:18.042629 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.042560    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"0979a61d8393f8b2f361037cc405788359d91d66289f93f6d142fc46e6df6a8e"}
Nov 04 00:59:18.042629 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.042593    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"854d4b95d2d337f0b9b9068ab7c29112aa355b7e3a26a973a85050f6b074bbc4"}
Nov 04 00:59:18.042629 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.042620    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"53c91a31f71533bb6080aa1dd503330ef2bcc9c27abe8cf998ea2658f13a3304"}
Nov 04 00:59:18.042786 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.042648    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7e75b897-7a6b-45d1-bfbd-2bc050c490d1","Type":"ContainerStarted","Data":"40a90261c92ee464992f75cb6d60680b4814f523b3ef0b3612603a04a9abfbe0"}
Nov 04 00:59:18.157627 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.157579    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-797775ffd6-zlbr7" podStartSLOduration=1.898960816 podStartE2EDuration="6.157564754s" totalImagesPullingTime="4.258603938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 00:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:59:18.156481174 +0000 UTC m=+126.818621238" watchObservedRunningTime="2026-11-04 00:59:18.157564754 +0000 UTC m=+126.819704856"
Nov 04 00:59:18.280822 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.280774    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.264024277 podStartE2EDuration="3.280760188s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.735911ms" isStatefulPod=false podCreationTimestamp="2026-11-04 00:59:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 00:59:18.279931966 +0000 UTC m=+126.942072027" watchObservedRunningTime="2026-11-04 00:59:18.280760188 +0000 UTC m=+126.942900249"
Nov 04 00:59:18.885680 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:18.885623    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:19.045265 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:19.045230    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:19.286683 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:19.286588    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:19.467800 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:19.467767    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:20.575326 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:20.575294    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 00:59:21.895182 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:21.895145    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 00:59:38.093643 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:38.093564    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 00:59:51.896198 ip-10-0-136-128 kubenswrapper[4434]: E1104 00:59:51.896159    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 00:59:55.434168 ip-10-0-136-128 kubenswrapper[4434]: I1104 00:59:55.434130    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:07.669660 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:07.669599    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:15.575075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:15.575030    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:00:15.590932 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:15.590905    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:00:16.264603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:16.264576    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 04 01:00:21.895594 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:00:21.895560    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:00:24.126722 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:24.126685    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:24.277152 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:24.277117    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:36.808403 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:36.808370    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:36.999363 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:36.999330    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:38.077557 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:38.077514    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:38.299594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:38.299562    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:39.341916 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:39.341884    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:39.496233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:39.496197    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:51.895089 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:00:51.895055    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:00:53.790707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:53.790619    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:53.951812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:53.951776    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:00:55.732897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:00:55.732863    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:01:21.895543 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:01:21.895498    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:01:51.896006 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:01:51.895960    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:01:56.114689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:01:56.114634    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:02:21.896095 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:02:21.896056    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:02:51.896060 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:02:51.895970    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:02:56.492080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:02:56.492029    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:12.185227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:12.185193    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:12.426514 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:12.426480    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:13.906393 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:13.906358    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:14.083949 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:14.083914    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:15.095393 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:15.095366    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:20.212417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:20.212376    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:20.453613 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:20.453541    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:21.412969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:21.412926    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:21.678359 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:21.678284    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:21.889456 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:21.889406    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:21.896241 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:03:21.896207    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:03:25.448508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:25.448469    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:30.323702 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:30.323648    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:03:51.895017 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:03:51.894980    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:03:56.764307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:03:56.764218    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:00.975802 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:00.975757    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:05.373424 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:05.373354    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:06.582711 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:06.582679    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:08.325201 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:08.325146    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:08.647448 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:08.647402    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:08.827141 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:08.827083    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:08.986859 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:08.986783    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:09.646110 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.646078    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"]
Nov 04 01:04:09.649294 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.649277    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.704056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.703981    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 04 01:04:09.745542 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.745508    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v94h2\" (UniqueName: \"kubernetes.io/projected/e83f961f-6e0a-47f8-800b-8b67df7c8c82-kube-api-access-v94h2\") pod \"cert-manager-webhook-5dbbbf75cc-88b27\" (UID: \"e83f961f-6e0a-47f8-800b-8b67df7c8c82\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.745542 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.745540    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e83f961f-6e0a-47f8-800b-8b67df7c8c82-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-88b27\" (UID: \"e83f961f-6e0a-47f8-800b-8b67df7c8c82\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.754329 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.754310    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 04 01:04:09.755446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.755429    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"]
Nov 04 01:04:09.846863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.846833    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v94h2\" (UniqueName: \"kubernetes.io/projected/e83f961f-6e0a-47f8-800b-8b67df7c8c82-kube-api-access-v94h2\") pod \"cert-manager-webhook-5dbbbf75cc-88b27\" (UID: \"e83f961f-6e0a-47f8-800b-8b67df7c8c82\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.846863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.846864    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e83f961f-6e0a-47f8-800b-8b67df7c8c82-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-88b27\" (UID: \"e83f961f-6e0a-47f8-800b-8b67df7c8c82\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.904066 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.904031    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e83f961f-6e0a-47f8-800b-8b67df7c8c82-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-88b27\" (UID: \"e83f961f-6e0a-47f8-800b-8b67df7c8c82\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.904203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.904180    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v94h2\" (UniqueName: \"kubernetes.io/projected/e83f961f-6e0a-47f8-800b-8b67df7c8c82-kube-api-access-v94h2\") pod \"cert-manager-webhook-5dbbbf75cc-88b27\" (UID: \"e83f961f-6e0a-47f8-800b-8b67df7c8c82\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:09.976115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:09.976044    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:10.349865 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:10.349781    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"]
Nov 04 01:04:10.360187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:10.359300    4434 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:04:10.663729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:10.663692    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:11.073386 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.073309    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27" event={"ID":"e83f961f-6e0a-47f8-800b-8b67df7c8c82","Type":"ContainerStarted","Data":"0a9f47268351cdc7e16fc75d598cb27e734e74df7ebf4fc86fe05911f5ac8fbd"}
Nov 04 01:04:11.354074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.353996    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"]
Nov 04 01:04:11.358185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.358167    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.457643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.457617    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22367ca7-ae96-4211-9443-650f0e17a601-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-b96ql\" (UID: \"22367ca7-ae96-4211-9443-650f0e17a601\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.457804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.457650    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pflkx\" (UniqueName: \"kubernetes.io/projected/22367ca7-ae96-4211-9443-650f0e17a601-kube-api-access-pflkx\") pod \"cert-manager-cainjector-5c57bbd4fc-b96ql\" (UID: \"22367ca7-ae96-4211-9443-650f0e17a601\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.461229 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.461207    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lzrf2\""
Nov 04 01:04:11.462345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.462322    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"]
Nov 04 01:04:11.558784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.558756    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22367ca7-ae96-4211-9443-650f0e17a601-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-b96ql\" (UID: \"22367ca7-ae96-4211-9443-650f0e17a601\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.558930 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.558788    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pflkx\" (UniqueName: \"kubernetes.io/projected/22367ca7-ae96-4211-9443-650f0e17a601-kube-api-access-pflkx\") pod \"cert-manager-cainjector-5c57bbd4fc-b96ql\" (UID: \"22367ca7-ae96-4211-9443-650f0e17a601\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.618520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.618444    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/22367ca7-ae96-4211-9443-650f0e17a601-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-b96ql\" (UID: \"22367ca7-ae96-4211-9443-650f0e17a601\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.618742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.618719    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pflkx\" (UniqueName: \"kubernetes.io/projected/22367ca7-ae96-4211-9443-650f0e17a601-kube-api-access-pflkx\") pod \"cert-manager-cainjector-5c57bbd4fc-b96ql\" (UID: \"22367ca7-ae96-4211-9443-650f0e17a601\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.669428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.669399    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"
Nov 04 01:04:11.757604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:11.757574    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:12.031655 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:12.031613    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:12.040286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:12.040257    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql"]
Nov 04 01:04:12.044527 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:12.044497    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22367ca7_ae96_4211_9443_650f0e17a601.slice/crio-0683a74e76f3cc779e510712a18c0a94e5743a74238e3346a84ff21877fc7234 WatchSource:0}: Error finding container 0683a74e76f3cc779e510712a18c0a94e5743a74238e3346a84ff21877fc7234: Status 404 returned error can't find the container with id 0683a74e76f3cc779e510712a18c0a94e5743a74238e3346a84ff21877fc7234
Nov 04 01:04:12.077384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:12.077346    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql" event={"ID":"22367ca7-ae96-4211-9443-650f0e17a601","Type":"ContainerStarted","Data":"0683a74e76f3cc779e510712a18c0a94e5743a74238e3346a84ff21877fc7234"}
Nov 04 01:04:12.205081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:12.205014    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:15.200289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:15.200198    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:15.383450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:15.383407    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:15.984764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:15.984738    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:16.100406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.100344    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27" event={"ID":"e83f961f-6e0a-47f8-800b-8b67df7c8c82","Type":"ContainerStarted","Data":"0aef7ada4cb29ae30b6b9d18a84614a9d7d6f62636ecc4941474f091d7517359"}
Nov 04 01:04:16.100586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.100483    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:16.215355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.215306    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27" podStartSLOduration=1.5980477770000001 podStartE2EDuration="7.215292903s" totalImagesPullingTime="5.617245126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:16.214838416 +0000 UTC m=+424.876978478" watchObservedRunningTime="2026-11-04 01:04:16.215292903 +0000 UTC m=+424.877432965"
Nov 04 01:04:16.549169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.549086    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:16.692356 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.692322    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"]
Nov 04 01:04:16.695790 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.695773    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.801308 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.801221    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-metrics-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.801308 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.801270    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-webhook-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.801515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.801374    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vvxn\" (UniqueName: \"kubernetes.io/projected/16e566d5-e973-4d84-9e89-98479ced6dc2-kube-api-access-6vvxn\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.801515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.801415    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.801515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.801472    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16e566d5-e973-4d84-9e89-98479ced6dc2-manifests\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.801515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.801512    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16e566d5-e973-4d84-9e89-98479ced6dc2-tmp\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.806005 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.805981    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 04 01:04:16.806005 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.805980    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 04 01:04:16.806165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.806034    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-j4czz\""
Nov 04 01:04:16.806165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.806086    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 04 01:04:16.806496 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.806481    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 04 01:04:16.806533 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.806507    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 04 01:04:16.811320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.811296    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:16.831493 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.831457    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"]
Nov 04 01:04:16.902132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902090    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-webhook-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.902305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902138    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vvxn\" (UniqueName: \"kubernetes.io/projected/16e566d5-e973-4d84-9e89-98479ced6dc2-kube-api-access-6vvxn\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.902305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902173    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.902305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902202    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16e566d5-e973-4d84-9e89-98479ced6dc2-manifests\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.902305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902239    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16e566d5-e973-4d84-9e89-98479ced6dc2-tmp\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.902305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902284    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-metrics-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.902925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.902876    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/16e566d5-e973-4d84-9e89-98479ced6dc2-manifests\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.904708 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.904655    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/16e566d5-e973-4d84-9e89-98479ced6dc2-tmp\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.904834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.904777    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-webhook-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.904926 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.904906    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.904988 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.904974    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/16e566d5-e973-4d84-9e89-98479ced6dc2-metrics-cert\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.962764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.962729    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vvxn\" (UniqueName: \"kubernetes.io/projected/16e566d5-e973-4d84-9e89-98479ced6dc2-kube-api-access-6vvxn\") pod \"opendatahub-operator-controller-manager-c655894cb-64knh\" (UID: \"16e566d5-e973-4d84-9e89-98479ced6dc2\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:16.985699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:16.985649    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:17.010692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.010650    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:17.106204 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.106164    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql" event={"ID":"22367ca7-ae96-4211-9443-650f0e17a601","Type":"ContainerStarted","Data":"4f05d6b878d8d7a7151d9544d5233033d5c24e0cbd98354cf00051a9fe7dcb42"}
Nov 04 01:04:17.213018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.212987    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:17.227539 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.227498    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b96ql" podStartSLOduration=2.004352431 podStartE2EDuration="6.227484442s" totalImagesPullingTime="4.223132011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:17.227245148 +0000 UTC m=+425.889385208" watchObservedRunningTime="2026-11-04 01:04:17.227484442 +0000 UTC m=+425.889624502"
Nov 04 01:04:17.456939 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.456899    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:17.532553 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.532521    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"]
Nov 04 01:04:17.536404 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:17.536376    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e566d5_e973_4d84_9e89_98479ced6dc2.slice/crio-49e482bcd0f4af51a846d9f836200303ba687f2a402acabb7b4b104746b3915a WatchSource:0}: Error finding container 49e482bcd0f4af51a846d9f836200303ba687f2a402acabb7b4b104746b3915a: Status 404 returned error can't find the container with id 49e482bcd0f4af51a846d9f836200303ba687f2a402acabb7b4b104746b3915a
Nov 04 01:04:17.652635 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.652604    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:17.920428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:17.920388    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:18.106508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:18.106474    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:18.110577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:18.110552    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh" event={"ID":"16e566d5-e973-4d84-9e89-98479ced6dc2","Type":"ContainerStarted","Data":"49e482bcd0f4af51a846d9f836200303ba687f2a402acabb7b4b104746b3915a"}
Nov 04 01:04:19.699577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:19.699530    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:21.255004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.254975    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:21.670206 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.670174    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-rz4qw"]
Nov 04 01:04:21.674577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.674560    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.735197 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.735170    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-rz4qw"]
Nov 04 01:04:21.745788 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.745765    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d667f9c4-51d1-46a7-8f48-39224f3ec8e1-bound-sa-token\") pod \"cert-manager-597b95cb84-rz4qw\" (UID: \"d667f9c4-51d1-46a7-8f48-39224f3ec8e1\") " pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.745924 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.745813    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpxcb\" (UniqueName: \"kubernetes.io/projected/d667f9c4-51d1-46a7-8f48-39224f3ec8e1-kube-api-access-tpxcb\") pod \"cert-manager-597b95cb84-rz4qw\" (UID: \"d667f9c4-51d1-46a7-8f48-39224f3ec8e1\") " pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.783198 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.783160    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wlslp\""
Nov 04 01:04:21.846451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.846412    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d667f9c4-51d1-46a7-8f48-39224f3ec8e1-bound-sa-token\") pod \"cert-manager-597b95cb84-rz4qw\" (UID: \"d667f9c4-51d1-46a7-8f48-39224f3ec8e1\") " pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.846593 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.846474    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpxcb\" (UniqueName: \"kubernetes.io/projected/d667f9c4-51d1-46a7-8f48-39224f3ec8e1-kube-api-access-tpxcb\") pod \"cert-manager-597b95cb84-rz4qw\" (UID: \"d667f9c4-51d1-46a7-8f48-39224f3ec8e1\") " pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.897115 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:04:21.896981    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:04:21.904870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.904848    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d667f9c4-51d1-46a7-8f48-39224f3ec8e1-bound-sa-token\") pod \"cert-manager-597b95cb84-rz4qw\" (UID: \"d667f9c4-51d1-46a7-8f48-39224f3ec8e1\") " pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.905814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.905795    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpxcb\" (UniqueName: \"kubernetes.io/projected/d667f9c4-51d1-46a7-8f48-39224f3ec8e1-kube-api-access-tpxcb\") pod \"cert-manager-597b95cb84-rz4qw\" (UID: \"d667f9c4-51d1-46a7-8f48-39224f3ec8e1\") " pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:21.988899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:21.988853    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-rz4qw"
Nov 04 01:04:22.108732 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:22.108702    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27"
Nov 04 01:04:22.128468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:22.128434    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh" event={"ID":"16e566d5-e973-4d84-9e89-98479ced6dc2","Type":"ContainerStarted","Data":"5168eef7c536bff53f5c922ad8184fa82da87a7830f844565be15af6392ba3d4"}
Nov 04 01:04:22.389069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:22.389032    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-rz4qw"]
Nov 04 01:04:22.392852 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:22.392823    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd667f9c4_51d1_46a7_8f48_39224f3ec8e1.slice/crio-52adaa3f00d2d2a3df9b81ab833c30ec84e4cb9251faea1dcd82aa7459bb848d WatchSource:0}: Error finding container 52adaa3f00d2d2a3df9b81ab833c30ec84e4cb9251faea1dcd82aa7459bb848d: Status 404 returned error can't find the container with id 52adaa3f00d2d2a3df9b81ab833c30ec84e4cb9251faea1dcd82aa7459bb848d
Nov 04 01:04:23.134233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:23.134189    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-rz4qw" event={"ID":"d667f9c4-51d1-46a7-8f48-39224f3ec8e1","Type":"ContainerStarted","Data":"60c7b6c90058c7c300b189b7c29efcbfa3c2ea42f1c5f2d4a706d3745e678ec7"}
Nov 04 01:04:23.134399 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:23.134244    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-rz4qw" event={"ID":"d667f9c4-51d1-46a7-8f48-39224f3ec8e1","Type":"ContainerStarted","Data":"52adaa3f00d2d2a3df9b81ab833c30ec84e4cb9251faea1dcd82aa7459bb848d"}
Nov 04 01:04:23.135726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:23.135702    4434 generic.go:356] "Generic (PLEG): container finished" podID="16e566d5-e973-4d84-9e89-98479ced6dc2" containerID="5168eef7c536bff53f5c922ad8184fa82da87a7830f844565be15af6392ba3d4" exitCode=0
Nov 04 01:04:23.135810 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:23.135745    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh" event={"ID":"16e566d5-e973-4d84-9e89-98479ced6dc2","Type":"ContainerDied","Data":"5168eef7c536bff53f5c922ad8184fa82da87a7830f844565be15af6392ba3d4"}
Nov 04 01:04:23.263073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:23.263023    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-rz4qw" podStartSLOduration=2.263010067 podStartE2EDuration="2.263010067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:23.261466182 +0000 UTC m=+431.923606243" watchObservedRunningTime="2026-11-04 01:04:23.263010067 +0000 UTC m=+431.925150127"
Nov 04 01:04:24.582227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:24.582124    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:24.906056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:24.906023    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:25.145688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:25.145639    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh" event={"ID":"16e566d5-e973-4d84-9e89-98479ced6dc2","Type":"ContainerStarted","Data":"b2c06e9defbdd4d6df7149c6ae786b1962e5ba30fff5c72295789e539c6f1504"}
Nov 04 01:04:25.145862 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:25.145797    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:25.491926 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:25.491815    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh" podStartSLOduration=3.789330878 podStartE2EDuration="9.491798037s" totalImagesPullingTime="5.236572178s" totalInitContainerRuntime="465.894981ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:25.447054494 +0000 UTC m=+434.109194556" watchObservedRunningTime="2026-11-04 01:04:25.491798037 +0000 UTC m=+434.153938099"
Nov 04 01:04:25.903925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:25.903888    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:26.098410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:26.098378    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:29.366367 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:29.366328    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:29.608422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:29.608379    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:29.916156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:29.916105    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:30.181482 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:30.181395    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:34.974592 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:34.974551    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:35.180242 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.180204    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:35.878907 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.878872    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"]
Nov 04 01:04:35.882364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.882335    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:35.894880 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.894854    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:35.935251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.935216    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"]
Nov 04 01:04:35.984398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.984358    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 04 01:04:35.984398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.984375    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Nov 04 01:04:35.984398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.984391    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Nov 04 01:04:35.984398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.984359    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 04 01:04:35.997725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:35.997698    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9g4nl\""
Nov 04 01:04:36.067109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.067063    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94adf17c-1b38-47b9-b02b-a668f858d8a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.067292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.067120    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwnkj\" (UniqueName: \"kubernetes.io/projected/94adf17c-1b38-47b9-b02b-a668f858d8a7-kube-api-access-mwnkj\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.067292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.067266    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94adf17c-1b38-47b9-b02b-a668f858d8a7-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.152024 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.151948    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c655894cb-64knh"
Nov 04 01:04:36.167863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.167832    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94adf17c-1b38-47b9-b02b-a668f858d8a7-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.167997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.167975    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94adf17c-1b38-47b9-b02b-a668f858d8a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.168040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.168009    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwnkj\" (UniqueName: \"kubernetes.io/projected/94adf17c-1b38-47b9-b02b-a668f858d8a7-kube-api-access-mwnkj\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.168596 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.168570    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/94adf17c-1b38-47b9-b02b-a668f858d8a7-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.170390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.170371    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/94adf17c-1b38-47b9-b02b-a668f858d8a7-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.223790 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.223754    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:36.257214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.257176    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwnkj\" (UniqueName: \"kubernetes.io/projected/94adf17c-1b38-47b9-b02b-a668f858d8a7-kube-api-access-mwnkj\") pod \"kuadrant-console-plugin-66c4fb575d-rhn29\" (UID: \"94adf17c-1b38-47b9-b02b-a668f858d8a7\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.509515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.509432    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"
Nov 04 01:04:36.889837 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:36.889800    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29"]
Nov 04 01:04:36.894838 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:36.894805    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod94adf17c_1b38_47b9_b02b_a668f858d8a7.slice/crio-d92da4f5ff6281924a61552315e656bfe0adec114d9ae98705603f2d78a04a6c WatchSource:0}: Error finding container d92da4f5ff6281924a61552315e656bfe0adec114d9ae98705603f2d78a04a6c: Status 404 returned error can't find the container with id d92da4f5ff6281924a61552315e656bfe0adec114d9ae98705603f2d78a04a6c
Nov 04 01:04:37.195144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:37.195063    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29" event={"ID":"94adf17c-1b38-47b9-b02b-a668f858d8a7","Type":"ContainerStarted","Data":"d92da4f5ff6281924a61552315e656bfe0adec114d9ae98705603f2d78a04a6c"}
Nov 04 01:04:37.694094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:37.694060    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:39.602355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:39.602323    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:39.767879 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:39.767841    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:45.232705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:45.232654    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29" event={"ID":"94adf17c-1b38-47b9-b02b-a668f858d8a7","Type":"ContainerStarted","Data":"56f42a3242e26757758e137f05b1f62e4b136c24bf508c54c683da980301d07c"}
Nov 04 01:04:45.349020 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:45.348968    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-rhn29" podStartSLOduration=2.8575268400000002 podStartE2EDuration="10.348955533s" totalImagesPullingTime="7.491428693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:45.347381238 +0000 UTC m=+454.009521311" watchObservedRunningTime="2026-11-04 01:04:45.348955533 +0000 UTC m=+454.011095625"
Nov 04 01:04:46.650875 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:46.650829    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:46.835074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:46.835032    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:47.670239 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:47.670201    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:47.869133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:47.869102    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:48.263679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:48.263645    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:49.863581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:49.863546    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:49.921545 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:49.921501    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"]
Nov 04 01:04:49.924842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:49.924818    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:49.979130 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:49.979092    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"]
Nov 04 01:04:50.027776 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.027743    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 04 01:04:50.027776 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.027783    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 04 01:04:50.028012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.027807    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-k5npg\""
Nov 04 01:04:50.028012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.027883    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 04 01:04:50.028012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.027917    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 04 01:04:50.080133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.080097    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:50.087184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.087152    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfd5d\" (UniqueName: \"kubernetes.io/projected/64ae051b-00f6-4d73-b124-5677d654f54e-kube-api-access-dfd5d\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.087361 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.087201    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64ae051b-00f6-4d73-b124-5677d654f54e-tmp\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.087439 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.087356    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64ae051b-00f6-4d73-b124-5677d654f54e-tls-certs\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.188090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.188050    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfd5d\" (UniqueName: \"kubernetes.io/projected/64ae051b-00f6-4d73-b124-5677d654f54e-kube-api-access-dfd5d\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.188090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.188093    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64ae051b-00f6-4d73-b124-5677d654f54e-tmp\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.188361 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.188127    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64ae051b-00f6-4d73-b124-5677d654f54e-tls-certs\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.190589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.190564    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64ae051b-00f6-4d73-b124-5677d654f54e-tmp\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.190848 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.190825    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/64ae051b-00f6-4d73-b124-5677d654f54e-tls-certs\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.250095 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.250053    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:50.250332 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.250311    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfd5d\" (UniqueName: \"kubernetes.io/projected/64ae051b-00f6-4d73-b124-5677d654f54e-kube-api-access-dfd5d\") pod \"kube-auth-proxy-5794fb467c-wv6mv\" (UID: \"64ae051b-00f6-4d73-b124-5677d654f54e\") " pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.498112 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.498019    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:50.535974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.535941    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"
Nov 04 01:04:50.664650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.664606    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:50.840528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.840428    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:50.906018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:50.905984    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv"]
Nov 04 01:04:50.910212 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:50.910183    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ae051b_00f6_4d73_b124_5677d654f54e.slice/crio-f624e98e738a64b281cb1c0c61b46ce74b4dae5ed2c8e753807600796f5793c2 WatchSource:0}: Error finding container f624e98e738a64b281cb1c0c61b46ce74b4dae5ed2c8e753807600796f5793c2: Status 404 returned error can't find the container with id f624e98e738a64b281cb1c0c61b46ce74b4dae5ed2c8e753807600796f5793c2
Nov 04 01:04:51.007450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:51.007417    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:51.203565 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:51.203532    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:51.258621 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:51.258588    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv" event={"ID":"64ae051b-00f6-4d73-b124-5677d654f54e","Type":"ContainerStarted","Data":"f624e98e738a64b281cb1c0c61b46ce74b4dae5ed2c8e753807600796f5793c2"}
Nov 04 01:04:51.896458 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:04:51.896417    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:04:52.510609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.510568    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:52.569982 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.569946    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:04:52.573602 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.573570    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.676865 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.676817    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 04 01:04:52.678436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.678411    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:04:52.712135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.712088    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file\") pod \"limitador-limitador-74dfcfb894-hlspf\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.712135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.712137    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvflk\" (UniqueName: \"kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk\") pod \"limitador-limitador-74dfcfb894-hlspf\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.812643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.812553    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:04:52.812873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.812853    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file\") pod \"limitador-limitador-74dfcfb894-hlspf\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.812917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.812884    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvflk\" (UniqueName: \"kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk\") pod \"limitador-limitador-74dfcfb894-hlspf\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.813550 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.813533    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file\") pod \"limitador-limitador-74dfcfb894-hlspf\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.871574 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.871538    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvflk\" (UniqueName: \"kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk\") pod \"limitador-limitador-74dfcfb894-hlspf\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:52.879041 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.879001    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:52.889996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:52.889966    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:53.057483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.057448    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:53.275295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.275265    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:04:53.279660 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:53.279630    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76e18fc_c2da_467c_9f95_dc1b55ebe316.slice/crio-cfe59e51269032d30b72a253ee04d52c464c0851aca823feb727531d2f34a7d3 WatchSource:0}: Error finding container cfe59e51269032d30b72a253ee04d52c464c0851aca823feb727531d2f34a7d3: Status 404 returned error can't find the container with id cfe59e51269032d30b72a253ee04d52c464c0851aca823feb727531d2f34a7d3
Nov 04 01:04:53.344894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.344861    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"]
Nov 04 01:04:53.348361 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.348344    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.393924 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.393890    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"]
Nov 04 01:04:53.417278 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.417246    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd66cd3c-fe47-4032-83fa-339b9dc95baf-metrics-certs\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.417446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.417314    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd66cd3c-fe47-4032-83fa-339b9dc95baf-cert\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.417446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.417422    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf75q\" (UniqueName: \"kubernetes.io/projected/fd66cd3c-fe47-4032-83fa-339b9dc95baf-kube-api-access-gf75q\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.417542 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.417511    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fd66cd3c-fe47-4032-83fa-339b9dc95baf-manager-config\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.427348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.427316    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 04 01:04:53.442914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.442884    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 04 01:04:53.442914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.442890    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-4hqhp\""
Nov 04 01:04:53.452055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.452021    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:53.453060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.453034    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 04 01:04:53.453194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.453167    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 04 01:04:53.453296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.453275    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 04 01:04:53.518018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.517980    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd66cd3c-fe47-4032-83fa-339b9dc95baf-cert\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.518018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.518016    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf75q\" (UniqueName: \"kubernetes.io/projected/fd66cd3c-fe47-4032-83fa-339b9dc95baf-kube-api-access-gf75q\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.518473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.518046    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fd66cd3c-fe47-4032-83fa-339b9dc95baf-manager-config\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.518473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.518099    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd66cd3c-fe47-4032-83fa-339b9dc95baf-metrics-certs\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.518842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.518816    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fd66cd3c-fe47-4032-83fa-339b9dc95baf-manager-config\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.520798 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.520767    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd66cd3c-fe47-4032-83fa-339b9dc95baf-metrics-certs\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.521029 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.521009    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd66cd3c-fe47-4032-83fa-339b9dc95baf-cert\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.575486 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.575393    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf75q\" (UniqueName: \"kubernetes.io/projected/fd66cd3c-fe47-4032-83fa-339b9dc95baf-kube-api-access-gf75q\") pod \"jobset-controller-manager-6685f7b4cc-pms8x\" (UID: \"fd66cd3c-fe47-4032-83fa-339b9dc95baf\") " pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:53.616196 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.616162    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:53.661431 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:53.661397    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:04:54.042329 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:54.042292    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"]
Nov 04 01:04:54.048097 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:04:54.048063    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd66cd3c_fe47_4032_83fa_339b9dc95baf.slice/crio-ec605164a606f497a4673d4153b4d160a6e5bcc7ad09147d254012e4281f2c55 WatchSource:0}: Error finding container ec605164a606f497a4673d4153b4d160a6e5bcc7ad09147d254012e4281f2c55: Status 404 returned error can't find the container with id ec605164a606f497a4673d4153b4d160a6e5bcc7ad09147d254012e4281f2c55
Nov 04 01:04:54.251886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:54.251856    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:54.274876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:54.274840    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf" event={"ID":"d76e18fc-c2da-467c-9f95-dc1b55ebe316","Type":"ContainerStarted","Data":"cfe59e51269032d30b72a253ee04d52c464c0851aca823feb727531d2f34a7d3"}
Nov 04 01:04:54.276793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:54.276765    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x" event={"ID":"fd66cd3c-fe47-4032-83fa-339b9dc95baf","Type":"ContainerStarted","Data":"ec605164a606f497a4673d4153b4d160a6e5bcc7ad09147d254012e4281f2c55"}
Nov 04 01:04:54.417605 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:54.417565    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:55.283331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:55.283292    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv" event={"ID":"64ae051b-00f6-4d73-b124-5677d654f54e","Type":"ContainerStarted","Data":"754bc780438508d36b8f2a6e0877f0979b5f2267a171259b724151fcf31a0821"}
Nov 04 01:04:55.398975 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:55.398916    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5794fb467c-wv6mv" podStartSLOduration=2.6875909399999998 podStartE2EDuration="6.39889998s" totalImagesPullingTime="3.71130904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:55.397758665 +0000 UTC m=+464.059898739" watchObservedRunningTime="2026-11-04 01:04:55.39889998 +0000 UTC m=+464.061040034"
Nov 04 01:04:57.124881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:57.124842    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:57.887528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:57.887490    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:58.301202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:58.301115    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf" event={"ID":"d76e18fc-c2da-467c-9f95-dc1b55ebe316","Type":"ContainerStarted","Data":"524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce"}
Nov 04 01:04:58.301642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:58.301219    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:04:58.419434 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:58.419375    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf" podStartSLOduration=1.698365334 podStartE2EDuration="6.419358066s" totalImagesPullingTime="4.720992732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:58.416792993 +0000 UTC m=+467.078933057" watchObservedRunningTime="2026-11-04 01:04:58.419358066 +0000 UTC m=+467.081498127"
Nov 04 01:04:59.306694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:59.306631    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x" event={"ID":"fd66cd3c-fe47-4032-83fa-339b9dc95baf","Type":"ContainerStarted","Data":"e41182346fc33eac2bc560bfba4aca518db8448151534b5dc01e5a1060d72c9a"}
Nov 04 01:04:59.426869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:59.426809    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x" podStartSLOduration=1.69843101 podStartE2EDuration="6.426795819s" totalImagesPullingTime="4.728364809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:04:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:04:59.425216943 +0000 UTC m=+468.087357007" watchObservedRunningTime="2026-11-04 01:04:59.426795819 +0000 UTC m=+468.088935858"
Nov 04 01:04:59.458447 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:59.458411    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:04:59.618749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:04:59.618635    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:00.311192 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:00.311164    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:05:06.480614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:06.480569    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:06.653713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:06.653653    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:09.309429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:09.309400    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:05:11.320280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:11.320253    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-6685f7b4cc-pms8x"
Nov 04 01:05:12.525265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:12.525231    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:20.074607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:20.074569    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:20.472088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:20.472050    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:20.699532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:20.699493    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:21.281958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:21.281915    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:21.896245 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:05:21.896203    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:05:35.423471 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:35.423386    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:05:35.423884 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:35.423614    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf" podUID="d76e18fc-c2da-467c-9f95-dc1b55ebe316" containerName="limitador" containerID="cri-o://524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce" gracePeriod=30
Nov 04 01:05:35.960729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:35.960704    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:05:36.094627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.094553    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk\" (UniqueName: \"kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk\") pod \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") "
Nov 04 01:05:36.094627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.094625    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file\" (UniqueName: \"kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file\") pod \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\" (UID: \"d76e18fc-c2da-467c-9f95-dc1b55ebe316\") "
Nov 04 01:05:36.095023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.094993    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file" pod "d76e18fc-c2da-467c-9f95-dc1b55ebe316" (UID: "d76e18fc-c2da-467c-9f95-dc1b55ebe316"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:05:36.096990 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.096967    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk" pod "d76e18fc-c2da-467c-9f95-dc1b55ebe316" (UID: "d76e18fc-c2da-467c-9f95-dc1b55ebe316"). InnerVolumeSpecName "kube-api-access-mvflk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:05:36.195925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.195891    4434 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d76e18fc-c2da-467c-9f95-dc1b55ebe316-config-file\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:05:36.195925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.195919    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvflk\" (UniqueName: \"kubernetes.io/projected/d76e18fc-c2da-467c-9f95-dc1b55ebe316-kube-api-access-mvflk\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:05:36.456741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.456702    4434 generic.go:356] "Generic (PLEG): container finished" podID="d76e18fc-c2da-467c-9f95-dc1b55ebe316" containerID="524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce" exitCode=0
Nov 04 01:05:36.457147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.456762    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf" event={"ID":"d76e18fc-c2da-467c-9f95-dc1b55ebe316","Type":"ContainerDied","Data":"524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce"}
Nov 04 01:05:36.457147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.456775    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf"
Nov 04 01:05:36.457147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.456843    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hlspf" event={"ID":"d76e18fc-c2da-467c-9f95-dc1b55ebe316","Type":"ContainerDied","Data":"cfe59e51269032d30b72a253ee04d52c464c0851aca823feb727531d2f34a7d3"}
Nov 04 01:05:36.457147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.456875    4434 scope.go:122] "RemoveContainer" containerID="524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce"
Nov 04 01:05:36.465240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.465220    4434 scope.go:122] "RemoveContainer" containerID="524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce"
Nov 04 01:05:36.465482 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:05:36.465460    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce\": container with ID starting with 524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce not found: ID does not exist" containerID="524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce"
Nov 04 01:05:36.465551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.465494    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce"} err="failed to get container status \"524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce\": rpc error: code = NotFound desc = could not find container \"524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce\": container with ID starting with 524ed1177921ea6faf5d23a36757a54632d7a2761f90d0781527a6408659dbce not found: ID does not exist"
Nov 04 01:05:36.620607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.620572    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:05:36.670840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:36.670812    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hlspf"]
Nov 04 01:05:37.471888 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:37.471858    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d76e18fc-c2da-467c-9f95-dc1b55ebe316" path="/var/lib/kubelet/pods/d76e18fc-c2da-467c-9f95-dc1b55ebe316/volumes"
Nov 04 01:05:41.760893 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:41.760859    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:41.931570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:41.931540    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:42.099261 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:42.099180    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:42.266012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:42.265977    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:46.558206 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:46.558172    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:46.719161 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:46.719125    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:47.736855 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:47.736817    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:49.404290 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:49.404260    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:49.597814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:49.597781    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:05:51.896370 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:05:51.896336    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:05:57.439713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:05:57.439681    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:00.773880 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:00.773846    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:12.460033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:12.459965    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:12.634837 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:12.634800    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:18.871995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:18.871952    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:19.137505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:19.137469    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:20.982600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:20.982558    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:21.197654 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:21.197599    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:21.433808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:21.433778    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:21.599084 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:21.599050    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:21.896459 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:06:21.896424    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:06:22.057507 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.057475    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"]
Nov 04 01:06:22.057915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.057890    4434 cpu_manager.go:414] "containerMap: removing container" podUID="d76e18fc-c2da-467c-9f95-dc1b55ebe316" containerName="limitador"
Nov 04 01:06:22.057915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.057902    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d76e18fc-c2da-467c-9f95-dc1b55ebe316" containerName="limitador"
Nov 04 01:06:22.057984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.057958    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="d76e18fc-c2da-467c-9f95-dc1b55ebe316" containerName="limitador"
Nov 04 01:06:22.062186 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.062169    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.117700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.117674    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:06:22.118599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.118565    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"]
Nov 04 01:06:22.166546 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.166462    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:06:22.168096 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.168078    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 01:06:22.168167 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.168078    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8zmjb\""
Nov 04 01:06:22.168167 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.168110    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 01:06:22.199954 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.199922    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.200092 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.199957    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.200132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.200093    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7dz7\" (UniqueName: \"kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.200132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.200123    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.300908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.300875    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7dz7\" (UniqueName: \"kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.301098 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.300920    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.301098 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.300960    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.301098 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.300979    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.301579 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.301556    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.303458 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.303437    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.303540 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.303503    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.359883 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.359851    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7dz7\" (UniqueName: \"kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7\") pod \"maas-api-6d669c7fcd-m5nxw\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.374725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.374692    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:22.749452 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:22.749377    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"]
Nov 04 01:06:22.753149 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:06:22.753122    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9783f6_ae10_4a8f_a13d_ffe023ac43d2.slice/crio-d6f0c81feb3e512d2796334cbc181e2954763785913b40e2d054023f332e028d WatchSource:0}: Error finding container d6f0c81feb3e512d2796334cbc181e2954763785913b40e2d054023f332e028d: Status 404 returned error can't find the container with id d6f0c81feb3e512d2796334cbc181e2954763785913b40e2d054023f332e028d
Nov 04 01:06:23.665299 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:23.665265    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw" event={"ID":"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2","Type":"ContainerStarted","Data":"d6f0c81feb3e512d2796334cbc181e2954763785913b40e2d054023f332e028d"}
Nov 04 01:06:25.199346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:25.199312    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:25.355597 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:25.355565    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:25.540069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:25.539987    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:25.709258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:25.709214    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:25.873991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:25.873963    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:26.037205 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.037165    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:26.166191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.166158    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"]
Nov 04 01:06:26.169928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.169907    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.226757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.226725    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"]
Nov 04 01:06:26.260059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.260032    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:26.275633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.275608    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 04 01:06:26.275866 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.275808    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-858n5\""
Nov 04 01:06:26.275932 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.275905    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 04 01:06:26.276704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.276687    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 04 01:06:26.338287 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.338258    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5j4\" (UniqueName: \"kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4\") pod \"llmisvc-controller-manager-869d5dcc5d-d6tkp\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.338413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.338319    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-d6tkp\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.439115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.439038    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5j4\" (UniqueName: \"kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4\") pod \"llmisvc-controller-manager-869d5dcc5d-d6tkp\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.439115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.439102    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-d6tkp\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.441398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.441378    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-d6tkp\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.496073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.496041    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5j4\" (UniqueName: \"kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4\") pod \"llmisvc-controller-manager-869d5dcc5d-d6tkp\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.680238 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.680203    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw" event={"ID":"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2","Type":"ContainerStarted","Data":"db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a"}
Nov 04 01:06:26.680423 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.680284    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:26.781704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.781605    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:26.797456 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:26.797085    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw" podStartSLOduration=1.491213097 podStartE2EDuration="4.797065159s" totalImagesPullingTime="3.305852062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:06:26.796301456 +0000 UTC m=+555.458441518" watchObservedRunningTime="2026-11-04 01:06:26.797065159 +0000 UTC m=+555.459205221"
Nov 04 01:06:27.153081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:27.153049    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"]
Nov 04 01:06:27.156724 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:06:27.156694    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod57105033_41af_4e71_bb3d_d3504ab4a2ea.slice/crio-7205539fa4566fe992ad9ee69d911499be431b3d3f6b77e24d412d0d28440b92 WatchSource:0}: Error finding container 7205539fa4566fe992ad9ee69d911499be431b3d3f6b77e24d412d0d28440b92: Status 404 returned error can't find the container with id 7205539fa4566fe992ad9ee69d911499be431b3d3f6b77e24d412d0d28440b92
Nov 04 01:06:27.350854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:27.350815    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:27.685917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:27.685881    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp" event={"ID":"57105033-41af-4e71-bb3d-d3504ab4a2ea","Type":"ContainerStarted","Data":"7205539fa4566fe992ad9ee69d911499be431b3d3f6b77e24d412d0d28440b92"}
Nov 04 01:06:30.701164 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:30.701118    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp" event={"ID":"57105033-41af-4e71-bb3d-d3504ab4a2ea","Type":"ContainerStarted","Data":"5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac"}
Nov 04 01:06:30.701534 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:30.701204    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:06:30.821194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:30.821093    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp" podStartSLOduration=1.462519317 podStartE2EDuration="4.821079969s" totalImagesPullingTime="3.358560652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:06:30.820311217 +0000 UTC m=+559.482451279" watchObservedRunningTime="2026-11-04 01:06:30.821079969 +0000 UTC m=+559.483220065"
Nov 04 01:06:32.074828 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.074795    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:32.227007 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.226977    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:32.377810 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.377777    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:32.528834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.528790    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:32.691627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.691545    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:32.730073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.730041    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:32.880111 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:32.880072    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:33.027186 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:33.027093    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:33.183365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:33.183333    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:33.346351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:33.346268    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:33.509498 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:33.509464    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:06:44.743832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:44.743798    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"]
Nov 04 01:06:44.744379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:44.744011    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw" podUID="6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" containerName="maas-api" containerID="cri-o://db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a" gracePeriod=30
Nov 04 01:06:44.986568 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:44.986543    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:45.101973 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.101896    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7\" (UniqueName: \"kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7\") pod \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") "
Nov 04 01:06:45.102151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.101993    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls\") pod \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") "
Nov 04 01:06:45.102151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.102020    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs\") pod \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") "
Nov 04 01:06:45.102151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.102067    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca\") pod \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\" (UID: \"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2\") "
Nov 04 01:06:45.103615 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.103580    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca" pod "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" (UID: "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:06:45.105040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.105010    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7" pod "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" (UID: "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2"). InnerVolumeSpecName "kube-api-access-g7dz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:06:45.105242 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.105215    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs" pod "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" (UID: "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:06:45.106192 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.106169    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls" pod "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" (UID: "6e9783f6-ae10-4a8f-a13d-ffe023ac43d2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:06:45.203554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.203509    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7dz7\" (UniqueName: \"kubernetes.io/projected/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-kube-api-access-g7dz7\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:06:45.203554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.203547    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:06:45.203554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.203560    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:06:45.203852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.203573    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:06:45.765194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.765102    4434 generic.go:356] "Generic (PLEG): container finished" podID="6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" containerID="db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a" exitCode=0
Nov 04 01:06:45.765194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.765146    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"
Nov 04 01:06:45.765194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.765185    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw" event={"ID":"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2","Type":"ContainerDied","Data":"db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a"}
Nov 04 01:06:45.765710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.765235    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw" event={"ID":"6e9783f6-ae10-4a8f-a13d-ffe023ac43d2","Type":"ContainerDied","Data":"d6f0c81feb3e512d2796334cbc181e2954763785913b40e2d054023f332e028d"}
Nov 04 01:06:45.765710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.765263    4434 scope.go:122] "RemoveContainer" containerID="db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a"
Nov 04 01:06:45.773563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.773543    4434 scope.go:122] "RemoveContainer" containerID="db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a"
Nov 04 01:06:45.773848 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:06:45.773829    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a\": container with ID starting with db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a not found: ID does not exist" containerID="db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a"
Nov 04 01:06:45.773907 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.773860    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a"} err="failed to get container status \"db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a\": rpc error: code = NotFound desc = could not find container \"db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a\": container with ID starting with db51c3472670f72c7444daa509a9220f6e96a78449f09b4a5098e8e0b22cf86a not found: ID does not exist"
Nov 04 01:06:45.929032 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.928994    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"]
Nov 04 01:06:45.932483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:45.932458    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-m5nxw"]
Nov 04 01:06:47.472826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:47.472791    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" path="/var/lib/kubelet/pods/6e9783f6-ae10-4a8f-a13d-ffe023ac43d2/volumes"
Nov 04 01:06:51.896334 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:06:51.896227    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:06:57.810216 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:06:57.810128    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:07:01.707433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:01.707398    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:07:21.896325 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:07:21.896287    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:07:39.799164 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.799128    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"]
Nov 04 01:07:39.799632 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.799480    4434 cpu_manager.go:414] "containerMap: removing container" podUID="6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" containerName="maas-api"
Nov 04 01:07:39.799632 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.799491    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" containerName="maas-api"
Nov 04 01:07:39.799632 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.799571    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e9783f6-ae10-4a8f-a13d-ffe023ac43d2" containerName="maas-api"
Nov 04 01:07:39.802588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.802573    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:39.857939 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.857910    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"]
Nov 04 01:07:39.977208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.977178    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg5kt\" (UniqueName: \"kubernetes.io/projected/3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c-kube-api-access-kg5kt\") pod \"llmisvc-controller-manager-6466c554d8-k8tl7\" (UID: \"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:39.977208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:39.977211    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c-cert\") pod \"llmisvc-controller-manager-6466c554d8-k8tl7\" (UID: \"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:40.078045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.077947    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg5kt\" (UniqueName: \"kubernetes.io/projected/3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c-kube-api-access-kg5kt\") pod \"llmisvc-controller-manager-6466c554d8-k8tl7\" (UID: \"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:40.078045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.077992    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c-cert\") pod \"llmisvc-controller-manager-6466c554d8-k8tl7\" (UID: \"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:40.080470 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.080434    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c-cert\") pod \"llmisvc-controller-manager-6466c554d8-k8tl7\" (UID: \"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:40.135861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.135823    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg5kt\" (UniqueName: \"kubernetes.io/projected/3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c-kube-api-access-kg5kt\") pod \"llmisvc-controller-manager-6466c554d8-k8tl7\" (UID: \"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:40.414604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.414569    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:40.784102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.784020    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"]
Nov 04 01:07:40.787602 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:07:40.787573    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd0cca4_6d58_4ea9_8d2b_b9df30eefc8c.slice/crio-fa62388d75f4c1c42a2c1758c3c0abfb7162b026e38d60247c1196533ceb2382 WatchSource:0}: Error finding container fa62388d75f4c1c42a2c1758c3c0abfb7162b026e38d60247c1196533ceb2382: Status 404 returned error can't find the container with id fa62388d75f4c1c42a2c1758c3c0abfb7162b026e38d60247c1196533ceb2382
Nov 04 01:07:40.998752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:40.998717    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7" event={"ID":"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c","Type":"ContainerStarted","Data":"fa62388d75f4c1c42a2c1758c3c0abfb7162b026e38d60247c1196533ceb2382"}
Nov 04 01:07:42.009871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:42.009834    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7" event={"ID":"3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c","Type":"ContainerStarted","Data":"878cd92b6c73db50636ab89785e746eab5cf98a85761b39e3db7dd6f044d3bf4"}
Nov 04 01:07:42.010279 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:42.009936    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:07:42.126333 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:42.125183    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7" podStartSLOduration=2.309733983 podStartE2EDuration="3.125164376s" totalImagesPullingTime="815.430393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:07:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:07:42.123634645 +0000 UTC m=+630.785774717" watchObservedRunningTime="2026-11-04 01:07:42.125164376 +0000 UTC m=+630.787304440"
Nov 04 01:07:51.896032 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:07:51.895995    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:07:58.151191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:07:58.151158    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:13.016490 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.016457    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-k8tl7"
Nov 04 01:08:13.183546 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.183512    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"]
Nov 04 01:08:13.183779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.183742    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp" podUID="57105033-41af-4e71-bb3d-d3504ab4a2ea" containerName="manager" containerID="cri-o://5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac" gracePeriod=30
Nov 04 01:08:13.428972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.428939    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:08:13.551123 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.551036    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4\" (UniqueName: \"kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4\") pod \"57105033-41af-4e71-bb3d-d3504ab4a2ea\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") "
Nov 04 01:08:13.551287 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.551153    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert\" (UniqueName: \"kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert\") pod \"57105033-41af-4e71-bb3d-d3504ab4a2ea\" (UID: \"57105033-41af-4e71-bb3d-d3504ab4a2ea\") "
Nov 04 01:08:13.553279 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.553250    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert" pod "57105033-41af-4e71-bb3d-d3504ab4a2ea" (UID: "57105033-41af-4e71-bb3d-d3504ab4a2ea"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:08:13.553385 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.553323    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4" pod "57105033-41af-4e71-bb3d-d3504ab4a2ea" (UID: "57105033-41af-4e71-bb3d-d3504ab4a2ea"). InnerVolumeSpecName "kube-api-access-lq5j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:08:13.652556 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.652519    4434 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57105033-41af-4e71-bb3d-d3504ab4a2ea-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:08:13.652556 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:13.652550    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq5j4\" (UniqueName: \"kubernetes.io/projected/57105033-41af-4e71-bb3d-d3504ab4a2ea-kube-api-access-lq5j4\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:08:14.108386 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.108353    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:14.147365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.147338    4434 generic.go:356] "Generic (PLEG): container finished" podID="57105033-41af-4e71-bb3d-d3504ab4a2ea" containerID="5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac" exitCode=0
Nov 04 01:08:14.147525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.147384    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"
Nov 04 01:08:14.147525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.147428    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp" event={"ID":"57105033-41af-4e71-bb3d-d3504ab4a2ea","Type":"ContainerDied","Data":"5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac"}
Nov 04 01:08:14.147525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.147485    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp" event={"ID":"57105033-41af-4e71-bb3d-d3504ab4a2ea","Type":"ContainerDied","Data":"7205539fa4566fe992ad9ee69d911499be431b3d3f6b77e24d412d0d28440b92"}
Nov 04 01:08:14.147525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.147512    4434 scope.go:122] "RemoveContainer" containerID="5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac"
Nov 04 01:08:14.156351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.156331    4434 scope.go:122] "RemoveContainer" containerID="5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac"
Nov 04 01:08:14.156622 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:08:14.156604    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac\": container with ID starting with 5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac not found: ID does not exist" containerID="5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac"
Nov 04 01:08:14.156709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.156632    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac"} err="failed to get container status \"5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac\": rpc error: code = NotFound desc = could not find container \"5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac\": container with ID starting with 5d9782cfcf6d922f4f96241ae0bf518938823f30dd7973d132a6d8af7ef29cac not found: ID does not exist"
Nov 04 01:08:14.326548 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.326514    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:14.332133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.332104    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"]
Nov 04 01:08:14.336389 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:14.336368    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-d6tkp"]
Nov 04 01:08:15.472297 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:15.472263    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57105033-41af-4e71-bb3d-d3504ab4a2ea" path="/var/lib/kubelet/pods/57105033-41af-4e71-bb3d-d3504ab4a2ea/volumes"
Nov 04 01:08:21.895973 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:08:21.895933    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:08:34.262963 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:34.262874    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:38.176636 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:38.176602    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:44.770640 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:44.770609    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:48.356711 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:48.356661    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:51.895317 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:08:51.895278    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:08:52.247991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:52.247884    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:56.456130 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:56.456084    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:08:58.464166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:08:58.464133    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:00.347330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:00.347289    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:04.744134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:04.744099    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:08.982506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:08.982465    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"]
Nov 04 01:09:08.983078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:08.982888    4434 cpu_manager.go:414] "containerMap: removing container" podUID="57105033-41af-4e71-bb3d-d3504ab4a2ea" containerName="manager"
Nov 04 01:09:08.983078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:08.982903    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57105033-41af-4e71-bb3d-d3504ab4a2ea" containerName="manager"
Nov 04 01:09:08.983078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:08.982969    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="57105033-41af-4e71-bb3d-d3504ab4a2ea" containerName="manager"
Nov 04 01:09:08.985622 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:08.985606    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.039998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.039951    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040003    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/464e35a4-a96d-4e48-a38a-9ef537538445-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040031    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040088    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/464e35a4-a96d-4e48-a38a-9ef537538445-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040140    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqqrg\" (UniqueName: \"kubernetes.io/projected/464e35a4-a96d-4e48-a38a-9ef537538445-kube-api-access-mqqrg\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040236    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040282    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040359 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040319    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.040359 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.040337    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/464e35a4-a96d-4e48-a38a-9ef537538445-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.092890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.092859    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 04 01:09:09.092890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.092859    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fwrps\""
Nov 04 01:09:09.100369 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.100341    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"]
Nov 04 01:09:09.141283 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141236    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/464e35a4-a96d-4e48-a38a-9ef537538445-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141283 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141293    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141314    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/464e35a4-a96d-4e48-a38a-9ef537538445-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141334    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqqrg\" (UniqueName: \"kubernetes.io/projected/464e35a4-a96d-4e48-a38a-9ef537538445-kube-api-access-mqqrg\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141383    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141412    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141455    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141486    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/464e35a4-a96d-4e48-a38a-9ef537538445-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141552    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.141958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141744    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.142387 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.141963    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.142387 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.142208    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/464e35a4-a96d-4e48-a38a-9ef537538445-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.142387 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.142275    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.142608 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.142591    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.143851 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.143831    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/464e35a4-a96d-4e48-a38a-9ef537538445-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.144074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.144055    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/464e35a4-a96d-4e48-a38a-9ef537538445-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.209650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.209614    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/464e35a4-a96d-4e48-a38a-9ef537538445-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.210088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.210065    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqqrg\" (UniqueName: \"kubernetes.io/projected/464e35a4-a96d-4e48-a38a-9ef537538445-kube-api-access-mqqrg\") pod \"maas-default-gateway-openshift-default-694db55c87-4q2t9\" (UID: \"464e35a4-a96d-4e48-a38a-9ef537538445\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.298010 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.297921    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:09.734006 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:09.733085    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"]
Nov 04 01:09:09.740352 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:09:09.740319    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464e35a4_a96d_4e48_a38a_9ef537538445.slice/crio-df0ed9f1a7a18412cc4064526eac3492016a8423d3a4352a684cce80233b11c2 WatchSource:0}: Error finding container df0ed9f1a7a18412cc4064526eac3492016a8423d3a4352a684cce80233b11c2: Status 404 returned error can't find the container with id df0ed9f1a7a18412cc4064526eac3492016a8423d3a4352a684cce80233b11c2
Nov 04 01:09:10.395264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:10.395224    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" event={"ID":"464e35a4-a96d-4e48-a38a-9ef537538445","Type":"ContainerStarted","Data":"df0ed9f1a7a18412cc4064526eac3492016a8423d3a4352a684cce80233b11c2"}
Nov 04 01:09:14.360723 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:14.360688    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:09:14.361055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:14.360758    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:09:14.361055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:14.360788    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:09:15.420476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:15.420441    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" event={"ID":"464e35a4-a96d-4e48-a38a-9ef537538445","Type":"ContainerStarted","Data":"3142789026d91f06fa4de6a9d45f23805e15e736b8293c76e69c57ee05a4ba38"}
Nov 04 01:09:15.641720 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:15.641650    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podStartSLOduration=3.023990549 podStartE2EDuration="7.641634507s" totalImagesPullingTime="4.617643958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:09:15.593073937 +0000 UTC m=+724.255213998" watchObservedRunningTime="2026-11-04 01:09:15.641634507 +0000 UTC m=+724.303774568"
Nov 04 01:09:16.299012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:16.298978    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:16.300387 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:16.300353    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-4q2t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 01:09:16.300519 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:16.300410    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podUID="464e35a4-a96d-4e48-a38a-9ef537538445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 01:09:17.298683 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:17.298627    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-4q2t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 01:09:17.299113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:17.298715    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podUID="464e35a4-a96d-4e48-a38a-9ef537538445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 01:09:18.298723 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:18.298688    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-4q2t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 01:09:18.299105 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:18.298742    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podUID="464e35a4-a96d-4e48-a38a-9ef537538445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 01:09:19.299150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:19.299112    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-4q2t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 01:09:19.299560 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:19.299169    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podUID="464e35a4-a96d-4e48-a38a-9ef537538445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 01:09:20.298857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:20.298810    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-4q2t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 01:09:20.299037 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:20.298884    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podUID="464e35a4-a96d-4e48-a38a-9ef537538445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 01:09:21.298554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:21.298517    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-4q2t9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 04 01:09:21.298976 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:21.298584    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9" podUID="464e35a4-a96d-4e48-a38a-9ef537538445" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 04 01:09:21.897555 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:09:21.897508    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:09:22.548094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:22.548065    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:22.548600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:22.548513    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:22.549364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:22.549338    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-4q2t9"
Nov 04 01:09:34.365228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:34.365195    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:34.550604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:34.550563    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:39.258149 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.258108    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"]
Nov 04 01:09:39.261986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.261962    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.342983 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.342949    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:39.353107 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.353073    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"]
Nov 04 01:09:39.418996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.418963    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419009    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419037    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419135    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419170    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419340 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419282    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419340 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419325    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419344    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.419407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.419362    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpbmq\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.514214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.514130    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:39.520348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520321    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520364    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520386    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520415    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520436    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520490    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520540    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520565    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520593    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpbmq\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520846    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.520928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520915    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.521034 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.520975    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.521193 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.521154    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.521374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.521355    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.523132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.523106    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.523286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.523268    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.577448 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.577418    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.577591 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.577500    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpbmq\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq\") pod \"maas-default-gateway-openshift-default-694db55c87-hcxkc\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:39.873680 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:39.873641    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:40.300143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.300059    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"]
Nov 04 01:09:40.303844 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:09:40.303815    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9f3f39_fdf6_41c9_943c_0f48fae93fe7.slice/crio-30a82158e6f03b1205f8256de1ad8d1c3a0598d35e392a457265304aaf4e70ef WatchSource:0}: Error finding container 30a82158e6f03b1205f8256de1ad8d1c3a0598d35e392a457265304aaf4e70ef: Status 404 returned error can't find the container with id 30a82158e6f03b1205f8256de1ad8d1c3a0598d35e392a457265304aaf4e70ef
Nov 04 01:09:40.306923 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.306893    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:09:40.306995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.306961    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:09:40.306995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.306990    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:09:40.531401 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.531364    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" event={"ID":"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7","Type":"ContainerStarted","Data":"810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22"}
Nov 04 01:09:40.531603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.531429    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" event={"ID":"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7","Type":"ContainerStarted","Data":"30a82158e6f03b1205f8256de1ad8d1c3a0598d35e392a457265304aaf4e70ef"}
Nov 04 01:09:40.654269 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.654217    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podStartSLOduration=1.6541983820000001 podStartE2EDuration="1.654198382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:09:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:09:40.652953763 +0000 UTC m=+749.315093824" watchObservedRunningTime="2026-11-04 01:09:40.654198382 +0000 UTC m=+749.316338442"
Nov 04 01:09:40.882656 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:40.882571    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:41.883692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:41.880559    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hcxkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:09:41.883692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:41.880627    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 01:09:42.888531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:42.888080    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hcxkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:09:42.888531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:42.888144    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 01:09:43.886688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:43.886108    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hcxkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:09:43.886688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:43.886173    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 01:09:44.881055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:44.881015    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hcxkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:09:44.881641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:44.881610    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 01:09:45.887423 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:45.887113    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hcxkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:09:45.887423 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:45.887183    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 01:09:46.874492 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:46.874459    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-hcxkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 04 01:09:46.874704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:46.874512    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 04 01:09:48.570739 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:48.570707    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:48.604484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:48.604450    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:48.605164 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:48.605140    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:48.605766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:48.605749    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:09:48.739577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:48.739541    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:09:51.896481 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:09:51.896441    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:09:58.801204 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:09:58.801105    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:12.123861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:12.123808    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:12.670464 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:12.670431    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:14.303982 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:14.303948    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:14.506377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:14.506343    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:16.076706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:16.076654    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:16.268144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:16.268092    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:21.896737 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:10:21.896695    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:10:42.860717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:42.860683    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:43.050604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:43.050572    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:43.581901 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:43.581859    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:43.886219 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:43.886187    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:44.060391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:44.060354    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:44.236259 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:44.236172    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:44.400164 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:44.400133    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:44.705063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:44.705025    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:44.873229 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:44.873166    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:45.046391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:45.046279    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:45.273241 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:45.273204    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:45.432974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:45.432942    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:45.635614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:45.635580    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:45.801934 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:45.801855    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:48.047831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:48.047798    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:48.229818 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:48.229786    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:48.400883 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:48.400853    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:48.721004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:48.720926    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:48.884660 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:48.884624    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:49.168083 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:49.168049    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:49.337517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:49.337480    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:49.841879 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:49.841844    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:50.076575 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:50.076542    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:50.245738 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:50.245706    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:50.520604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:50.520517    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:50.713120 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:50.713086    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:50.886847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:50.886813    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:51.896630 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:10:51.896587    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:10:52.294762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:52.294648    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:52.471749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:52.471712    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:52.634634 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:52.634601    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:52.939159 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:52.939086    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:53.112799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:53.112767    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:54.063571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:54.063537    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:54.287320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:54.287286    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:54.462533 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:54.462505    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:54.625123 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:54.625084    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:54.784601 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:54.784505    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:54.965991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:54.965958    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:55.138865 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:55.138833    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:57.758361 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:57.758330    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:57.927749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:57.927699    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:59.135853 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.135819    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:10:59.759699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.759652    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"]
Nov 04 01:10:59.763246 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.763224    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.816366 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.816334    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:10:59.866728 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.866696    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-e993d7-serving-cert\""
Nov 04 01:10:59.866728 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.866725    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 01:10:59.866948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.866695    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:10:59.866948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.866769    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8zmjb\""
Nov 04 01:10:59.869611 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.869584    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"]
Nov 04 01:10:59.871242 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.871220    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.871400 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.871379    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.871458 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.871418    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29v6q\" (UniqueName: \"kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.871551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.871534    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.972817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.972784    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.972817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.972822    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29v6q\" (UniqueName: \"kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.973048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.972871    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.973048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.972904    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.973048 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:10:59.972935    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-serving-cert: secret "maas-api-e2e-worker-w0-e993d7-serving-cert" not found
Nov 04 01:10:59.973048 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:10:59.973022    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls podName:7b514c36-7b50-44cd-bc14-e1d19296829b nodeName:}" failed. No retries permitted until 2026-11-04 01:11:00.473000184 +0000 UTC m=+829.135140237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls") pod "maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" (UID: "7b514c36-7b50-44cd-bc14-e1d19296829b") : secret "maas-api-e2e-worker-w0-e993d7-serving-cert" not found
Nov 04 01:10:59.973597 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.973575    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.975323 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.975301    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:10:59.982053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:10:59.982011    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:00.037275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.037203    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29v6q\" (UniqueName: \"kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:11:00.152191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.152153    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:00.314552 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.314465    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:00.471701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.471641    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:00.476881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.476854    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:11:00.479262 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.479239    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:11:00.675495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:00.675462    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:11:01.045811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:01.045731    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"]
Nov 04 01:11:01.049922 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:11:01.049894    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b514c36_7b50_44cd_bc14_e1d19296829b.slice/crio-741169c52ea75a40085c150c7788f86706fc3634612ef02c6e137733179ec0de WatchSource:0}: Error finding container 741169c52ea75a40085c150c7788f86706fc3634612ef02c6e137733179ec0de: Status 404 returned error can't find the container with id 741169c52ea75a40085c150c7788f86706fc3634612ef02c6e137733179ec0de
Nov 04 01:11:01.051218 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:01.051200    4434 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:11:01.872279 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:01.872234    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" event={"ID":"7b514c36-7b50-44cd-bc14-e1d19296829b","Type":"ContainerStarted","Data":"741169c52ea75a40085c150c7788f86706fc3634612ef02c6e137733179ec0de"}
Nov 04 01:11:02.878104 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:02.878066    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" event={"ID":"7b514c36-7b50-44cd-bc14-e1d19296829b","Type":"ContainerStarted","Data":"a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97"}
Nov 04 01:11:02.878613 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:02.878178    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:11:03.008537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:03.008477    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" podStartSLOduration=3.111310792 podStartE2EDuration="4.008462411s" totalImagesPullingTime="897.151619ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:03.005628355 +0000 UTC m=+831.667768417" watchObservedRunningTime="2026-11-04 01:11:03.008462411 +0000 UTC m=+831.670602504"
Nov 04 01:11:05.262642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.262602    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:05.347583 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.347549    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"]
Nov 04 01:11:05.351908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.351882    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.406121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.406088    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"]
Nov 04 01:11:05.422957 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.422918    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqmrz\" (UniqueName: \"kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.423145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.422976    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.423145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.423049    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.423145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.423102    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.444233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.444203    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:05.452228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.452206    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-678aaf-serving-cert\""
Nov 04 01:11:05.523835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.523747    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqmrz\" (UniqueName: \"kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.523835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.523787    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.523835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.523807    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.524139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.523841    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.524139 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:11:05.523964    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-serving-cert: secret "maas-api-e2e-worker-w6-678aaf-serving-cert" not found
Nov 04 01:11:05.524139 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:11:05.524040    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls podName:e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0 nodeName:}" failed. No retries permitted until 2026-11-04 01:11:06.024015212 +0000 UTC m=+834.686155277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls") pod "maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" (UID: "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0") : secret "maas-api-e2e-worker-w6-678aaf-serving-cert" not found
Nov 04 01:11:05.524456 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.524437    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.526238 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.526214    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.581618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.581588    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqmrz\" (UniqueName: \"kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:05.604950 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.604914    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:05.808894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.808806    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:05.974088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:05.974056    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:06.029311 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:06.029271    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:06.031776 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:06.031751    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\") pod \"maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:06.268368 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:06.268340    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:06.647042 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:06.647010    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"]
Nov 04 01:11:06.651008 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:11:06.650981    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e50c1c_8e1c_495c_8373_4ea55cfcc0c0.slice/crio-1053c8d8a9c6c351300f7101e585d33a6b06a2c2d6b2d8a2b7556009c18fb994 WatchSource:0}: Error finding container 1053c8d8a9c6c351300f7101e585d33a6b06a2c2d6b2d8a2b7556009c18fb994: Status 404 returned error can't find the container with id 1053c8d8a9c6c351300f7101e585d33a6b06a2c2d6b2d8a2b7556009c18fb994
Nov 04 01:11:06.898054 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:06.897975    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" event={"ID":"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0","Type":"ContainerStarted","Data":"1053c8d8a9c6c351300f7101e585d33a6b06a2c2d6b2d8a2b7556009c18fb994"}
Nov 04 01:11:07.903911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:07.903878    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" event={"ID":"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0","Type":"ContainerStarted","Data":"52421b6914aed01ce4e1e92c263a3ca456dcdc3fbd9e1ca7db69018ac8f78ed5"}
Nov 04 01:11:07.904344 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:07.904001    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:08.020051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:08.019994    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" podStartSLOduration=2.08931905 podStartE2EDuration="3.01998191s" totalImagesPullingTime="930.66286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:08.019019954 +0000 UTC m=+836.681160014" watchObservedRunningTime="2026-11-04 01:11:08.01998191 +0000 UTC m=+836.682121970"
Nov 04 01:11:08.888764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:08.888736    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:11:10.258735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:10.258697    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:10.434826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:10.434795    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:10.889147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:10.889111    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:12.844004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:12.843973    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:13.025465 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:13.025432    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:13.914852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:13.914819    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:11:15.406554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.406521    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:15.599517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.599488    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:15.755929 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.755841    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:15.791263 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.791233    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"]
Nov 04 01:11:15.795051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.795033    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:15.882360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.882329    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-2ee045\""
Nov 04 01:11:15.900325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.900298    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-2ee045-dockercfg-m54kt\""
Nov 04 01:11:15.901899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.901879    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"]
Nov 04 01:11:15.914516 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.914494    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:15.921747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.921721    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8n2s\" (UniqueName: \"kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s\") pod \"payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:15.921840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:15.921771    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:16.023332 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.023250    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:16.023500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.023348    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8n2s\" (UniqueName: \"kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s\") pod \"payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:16.024253 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.024228    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:16.080392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.080358    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8n2s\" (UniqueName: \"kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s\") pod \"payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:16.083455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.083432    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:16.107448 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.107418    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:16.477206 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.477166    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"]
Nov 04 01:11:16.480994 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:11:16.480958    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc5756b_93da_49c1_af2b_e0f5e341609d.slice/crio-400fbd22f4f31732f87e46311e2963505a588e2022568390a67a64b1f9f5fb73 WatchSource:0}: Error finding container 400fbd22f4f31732f87e46311e2963505a588e2022568390a67a64b1f9f5fb73: Status 404 returned error can't find the container with id 400fbd22f4f31732f87e46311e2963505a588e2022568390a67a64b1f9f5fb73
Nov 04 01:11:16.945800 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:16.945769    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" event={"ID":"2fc5756b-93da-49c1-af2b-e0f5e341609d","Type":"ContainerStarted","Data":"400fbd22f4f31732f87e46311e2963505a588e2022568390a67a64b1f9f5fb73"}
Nov 04 01:11:19.962508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:19.962473    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" event={"ID":"2fc5756b-93da-49c1-af2b-e0f5e341609d","Type":"ContainerStarted","Data":"f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef"}
Nov 04 01:11:19.962894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:19.962600    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:20.077862 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:20.077803    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" podStartSLOduration=2.238630838 podStartE2EDuration="5.07778961s" totalImagesPullingTime="2.839158772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:20.075961885 +0000 UTC m=+848.738101947" watchObservedRunningTime="2026-11-04 01:11:20.07778961 +0000 UTC m=+848.739929671"
Nov 04 01:11:20.873846 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:20.873810    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:21.896887 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:11:21.896848    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:11:25.600013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.599940    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"]
Nov 04 01:11:25.603680 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.603646    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.695091 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.695054    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbt5d\" (UniqueName: \"kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.695251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.695097    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.695251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.695209    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.695322 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.695254    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.706625 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.706608    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-c563ee-serving-cert\""
Nov 04 01:11:25.708252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.708220    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"]
Nov 04 01:11:25.795726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.795688    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbt5d\" (UniqueName: \"kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.795898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.795741    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.795898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.795779    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.795898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.795805    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.796073 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:11:25.795894    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-serving-cert: secret "maas-api-e2e-worker-w1-c563ee-serving-cert" not found
Nov 04 01:11:25.796073 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:11:25.795975    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls podName:3439c0be-bdb1-449e-a12c-d7553ec0d88e nodeName:}" failed. No retries permitted until 2026-11-04 01:11:26.295952816 +0000 UTC m=+854.958092874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls") pod "maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" (UID: "3439c0be-bdb1-449e-a12c-d7553ec0d88e") : secret "maas-api-e2e-worker-w1-c563ee-serving-cert" not found
Nov 04 01:11:25.796506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.796486    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.798045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.798027    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.818606 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.818578    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:25.856243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.856169    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbt5d\" (UniqueName: \"kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:25.996213 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:25.996184    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:26.161816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:26.161786    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:26.299622 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:26.299592    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:26.301861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:26.301833    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\") pod \"maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:26.516053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:26.515957    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:26.886563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:26.886531    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"]
Nov 04 01:11:26.889701 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:11:26.889661    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3439c0be_bdb1_449e_a12c_d7553ec0d88e.slice/crio-fbe1441a6339635191a04ef3a1ed62479207aad384e6d016bb52cfc43b735749 WatchSource:0}: Error finding container fbe1441a6339635191a04ef3a1ed62479207aad384e6d016bb52cfc43b735749: Status 404 returned error can't find the container with id fbe1441a6339635191a04ef3a1ed62479207aad384e6d016bb52cfc43b735749
Nov 04 01:11:26.994594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:26.994561    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" event={"ID":"3439c0be-bdb1-449e-a12c-d7553ec0d88e","Type":"ContainerStarted","Data":"fbe1441a6339635191a04ef3a1ed62479207aad384e6d016bb52cfc43b735749"}
Nov 04 01:11:28.000679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:28.000630    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" event={"ID":"3439c0be-bdb1-449e-a12c-d7553ec0d88e","Type":"ContainerStarted","Data":"713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae"}
Nov 04 01:11:28.001084 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:28.000763    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:28.115622 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:28.115508    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" podStartSLOduration=2.223650776 podStartE2EDuration="3.115494751s" totalImagesPullingTime="891.843975ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:28.114940947 +0000 UTC m=+856.777081285" watchObservedRunningTime="2026-11-04 01:11:28.115494751 +0000 UTC m=+856.777634812"
Nov 04 01:11:28.978639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:28.978599    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:29.586283 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:29.586240    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:29.755026 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:29.754993    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:30.969263 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:30.969229    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:11:31.039694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.039653    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"]
Nov 04 01:11:31.043461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.043445    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.075678 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.075642    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"]
Nov 04 01:11:31.095656 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.095622    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-e65b23-serving-cert\""
Nov 04 01:11:31.146904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.146864    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.147080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.146921    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.147150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.147078    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6p22\" (UniqueName: \"kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.147150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.147133    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.241813 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.241723    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:31.247966 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.247944    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.248090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.247986    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6p22\" (UniqueName: \"kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.248090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.248015    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.248193 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.248105    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.248772 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.248748    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.250472 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.250448    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.250568 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.250525    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.306161 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.306129    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6p22\" (UniqueName: \"kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22\") pod \"maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.356380 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.356349    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:31.398392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.398361    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:31.671546 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.671509    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:31.737350 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:31.737308    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"]
Nov 04 01:11:31.742009 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:11:31.741981    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae109d0f_8ced_48dd_892b_25fb80868747.slice/crio-c76a9266da2777e7fda740c9885925bfd4d422c44eab170dbe1eca348856d024 WatchSource:0}: Error finding container c76a9266da2777e7fda740c9885925bfd4d422c44eab170dbe1eca348856d024: Status 404 returned error can't find the container with id c76a9266da2777e7fda740c9885925bfd4d422c44eab170dbe1eca348856d024
Nov 04 01:11:32.020435 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:32.020345    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs" event={"ID":"ae109d0f-8ced-48dd-892b-25fb80868747","Type":"ContainerStarted","Data":"c76a9266da2777e7fda740c9885925bfd4d422c44eab170dbe1eca348856d024"}
Nov 04 01:11:33.026056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:33.025968    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs" event={"ID":"ae109d0f-8ced-48dd-892b-25fb80868747","Type":"ContainerStarted","Data":"3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d"}
Nov 04 01:11:33.026457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:33.026124    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:33.141996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:33.141946    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs" podStartSLOduration=2.261116998 podStartE2EDuration="3.14193277s" totalImagesPullingTime="880.815772ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:33.141463098 +0000 UTC m=+861.803603172" watchObservedRunningTime="2026-11-04 01:11:33.14193277 +0000 UTC m=+861.804072830"
Nov 04 01:11:34.010838 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.010810    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:11:34.782733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.782695    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"]
Nov 04 01:11:34.786398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.786379    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.839882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.839852    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-2fd27d\"/\"openshift-service-ca.crt\""
Nov 04 01:11:34.884134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.884104    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.884284 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.884144    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.884284 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.884271    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbxlp\" (UniqueName: \"kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.884365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.884312    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.884365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.884355    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.884428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.884372    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.890014 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.889988    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-2fd27d\"/\"simulator-w2-2fd27d-kserve-self-signed-certs\""
Nov 04 01:11:34.890014 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.890008    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-2fd27d\"/\"kube-root-ca.crt\""
Nov 04 01:11:34.890195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.890051    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-2fd27d\"/\"default-dockercfg-kvnqv\""
Nov 04 01:11:34.891824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.891797    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"]
Nov 04 01:11:34.912161 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.912137    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:34.985810 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.985781    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.985953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.985842    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.985953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.985932    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.986023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.985970    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.986023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.986004    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.986101 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.986049    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbxlp\" (UniqueName: \"kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.986303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.986282    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.986379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.986359    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.986508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.986478    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.988236 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.988202    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:34.988436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:34.988418    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:35.044088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:35.044016    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbxlp\" (UniqueName: \"kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp\") pod \"simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") " pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:35.097950 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:35.097908    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:35.477228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:35.477195    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"]
Nov 04 01:11:35.480915 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:11:35.480886    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd96fe731_d4cd_4c29_86a4_1e6175b0b7a8.slice/crio-d8fcb001c57612ecd2fb3777c64ffbf0ba532323b19eb3b466217b6e471389d3 WatchSource:0}: Error finding container d8fcb001c57612ecd2fb3777c64ffbf0ba532323b19eb3b466217b6e471389d3: Status 404 returned error can't find the container with id d8fcb001c57612ecd2fb3777c64ffbf0ba532323b19eb3b466217b6e471389d3
Nov 04 01:11:36.040579 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:36.040542    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" event={"ID":"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8","Type":"ContainerStarted","Data":"d8fcb001c57612ecd2fb3777c64ffbf0ba532323b19eb3b466217b6e471389d3"}
Nov 04 01:11:36.512093 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:36.512049    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:38.051562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:38.051526    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" event={"ID":"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8","Type":"ContainerStarted","Data":"d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3"}
Nov 04 01:11:38.098107 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:38.098078    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:38.167821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:38.167763    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podStartSLOduration=2.337364946 podStartE2EDuration="4.16774869s" totalImagesPullingTime="1.830383744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:11:38.166815735 +0000 UTC m=+866.828955795" watchObservedRunningTime="2026-11-04 01:11:38.16774869 +0000 UTC m=+866.829888751"
Nov 04 01:11:39.035498 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:39.035466    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:11:41.465558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:41.465525    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:41.625150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:41.625103    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:41.783807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:41.783723    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:44.648128 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:44.648096    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:45.099132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:45.099048    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:45.111850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:45.111818    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:45.232124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:45.232089    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:45.400921 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:45.400890    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:45.584474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:45.584420    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:46.098345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:46.098317    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:11:47.152543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:47.152511    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:51.897292 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:11:51.897192    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:11:52.882519 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:52.882480    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:54.917365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:54.917318    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:55.508220 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:55.508186    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:55.676489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:55.676456    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:55.868326 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:55.868238    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:58.182571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:58.182529    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:11:59.473410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:11:59.473374    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:03.441473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:03.441432    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:03.997450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:03.997420    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"]
Nov 04 01:12:04.001183 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.001168    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.104868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.104831    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e993d7\"/\"simulator-w0-e993d7-kserve-self-signed-certs\""
Nov 04 01:12:04.104868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.104831    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e993d7\"/\"kube-root-ca.crt\""
Nov 04 01:12:04.104868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.104829    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e993d7\"/\"default-dockercfg-cljcx\""
Nov 04 01:12:04.105166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.104856    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e993d7\"/\"openshift-service-ca.crt\""
Nov 04 01:12:04.106657 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.106635    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"]
Nov 04 01:12:04.114067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.114037    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:04.157570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.157542    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.157730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.157602    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.157730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.157720    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.157820 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.157741    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.157820 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.157765    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.157895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.157833    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkhtd\" (UniqueName: \"kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.258607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.258530    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.258607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.258567    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.258607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.258588    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.258882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.258614    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkhtd\" (UniqueName: \"kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.258882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.258699    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.258882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.258746    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.259092 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.259067    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.259156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.259139    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.259156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.259143    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.260963 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.260936    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.261145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.261126    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.293004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.292976    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:04.316020 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.315998    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkhtd\" (UniqueName: \"kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd\") pod \"simulator-w0-e993d7-kserve-6b95cc586b-kzr7w\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") " pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.613019 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.612935    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:04.935230 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:04.935174    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:05.000317 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:05.000288    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"]
Nov 04 01:12:05.004185 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:12:05.004150    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea69d046_a993_4458_a52a_b30a2ff580d7.slice/crio-a3e5f3dad2649c5ff932b232a8f075e318622a807f229252976be35d22b97085 WatchSource:0}: Error finding container a3e5f3dad2649c5ff932b232a8f075e318622a807f229252976be35d22b97085: Status 404 returned error can't find the container with id a3e5f3dad2649c5ff932b232a8f075e318622a807f229252976be35d22b97085
Nov 04 01:12:05.116483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:05.116448    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:05.174756 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:05.174718    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" event={"ID":"ea69d046-a993-4458-a52a-b30a2ff580d7","Type":"ContainerStarted","Data":"1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0"}
Nov 04 01:12:05.174942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:05.174765    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" event={"ID":"ea69d046-a993-4458-a52a-b30a2ff580d7","Type":"ContainerStarted","Data":"a3e5f3dad2649c5ff932b232a8f075e318622a807f229252976be35d22b97085"}
Nov 04 01:12:05.292293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:05.292189    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podStartSLOduration=2.29217284 podStartE2EDuration="2.29217284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:05.290960308 +0000 UTC m=+893.953100368" watchObservedRunningTime="2026-11-04 01:12:05.29217284 +0000 UTC m=+893.954312901"
Nov 04 01:12:05.613121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:05.613049    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:08.878647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:08.878612    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:14.073629 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:14.073594    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:14.613283 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:14.613242    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:14.625554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:14.625524    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:15.231489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:15.231459    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:12:16.082841 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.082802    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"]
Nov 04 01:12:16.088847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.088822    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.141450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.141417    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"]
Nov 04 01:12:16.171810 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.171779    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.171984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.171824    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.171984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.171846    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss678\" (UniqueName: \"kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.171984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.171968    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.172137 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.172013    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.172137 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.172130    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.188727 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.188697    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-678aaf\"/\"kube-root-ca.crt\""
Nov 04 01:12:16.188853 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.188765    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-678aaf\"/\"openshift-service-ca.crt\""
Nov 04 01:12:16.196059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.196030    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-678aaf\"/\"default-dockercfg-wmh77\""
Nov 04 01:12:16.196160 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.196061    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-678aaf\"/\"simulator-w6-678aaf-kserve-self-signed-certs\""
Nov 04 01:12:16.205857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.205792    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:16.272641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.272611    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.272653    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss678\" (UniqueName: \"kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.272717    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.272987    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.273057    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.273128    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.273192    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273402 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.273310    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.273468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.273459    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.275038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.275012    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.275189 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.275172    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.330750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.330718    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss678\" (UniqueName: \"kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678\") pod \"simulator-w6-678aaf-kserve-7dfcc58ccb-w646m\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") " pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.381149 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.381102    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:16.401223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.401194    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:16.794556 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.794488    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"]
Nov 04 01:12:16.798306 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:12:16.798280    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9112be4e_695a_4087_a364_fd290ae66db3.slice/crio-1b2831b534c48608357392707c3f159513d62db3019c9a603893f08e0d0290ef WatchSource:0}: Error finding container 1b2831b534c48608357392707c3f159513d62db3019c9a603893f08e0d0290ef: Status 404 returned error can't find the container with id 1b2831b534c48608357392707c3f159513d62db3019c9a603893f08e0d0290ef
Nov 04 01:12:16.950941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:16.950896    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:17.142100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:17.142069    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:17.231964 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:17.231932    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" event={"ID":"9112be4e-695a-4087-a364-fd290ae66db3","Type":"ContainerStarted","Data":"39a8834311af0a39e30239583c521a7e6d0740ed13311f08313ea13ca072caeb"}
Nov 04 01:12:17.232159 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:17.231971    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" event={"ID":"9112be4e-695a-4087-a364-fd290ae66db3","Type":"ContainerStarted","Data":"1b2831b534c48608357392707c3f159513d62db3019c9a603893f08e0d0290ef"}
Nov 04 01:12:17.352939 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:17.352883    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podStartSLOduration=1.352866022 podStartE2EDuration="1.352866022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:17.350927735 +0000 UTC m=+906.013067799" watchObservedRunningTime="2026-11-04 01:12:17.352866022 +0000 UTC m=+906.015006083"
Nov 04 01:12:17.402118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:17.402040    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:19.210763 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:19.210713    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:21.896422 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:12:21.896389    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:12:21.951014 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:21.950979    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:22.575904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:22.575866    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:22.756792 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:22.756750    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:24.530737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:24.530703    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:26.402407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:26.402371    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:26.414819 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:26.414793    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:27.285827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:27.285794    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:12:29.212734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:29.212678    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:29.371925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:29.371875    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:29.531124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:29.531008    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:32.337720 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:32.337650    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:32.527457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:32.527418    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:33.022566 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:33.022532    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:33.180263 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:33.180228    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:34.571859 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:34.571828    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:36.143320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.143283    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"]
Nov 04 01:12:36.147233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.147215    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.243069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.243033    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e210ef\"/\"kube-root-ca.crt\""
Nov 04 01:12:36.252348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.252324    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e210ef\"/\"default-dockercfg-mblzv\""
Nov 04 01:12:36.252517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.252375    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e210ef\"/\"simulator-w5-e210ef-kserve-self-signed-certs\""
Nov 04 01:12:36.252517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.252383    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e210ef\"/\"openshift-service-ca.crt\""
Nov 04 01:12:36.254460 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.254434    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"]
Nov 04 01:12:36.268766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.268742    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.268883 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.268780    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.268985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.268963    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzwsw\" (UniqueName: \"kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.269035 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.269001    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.269035 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.269022    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.269225 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.269192    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.274115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.274096    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:36.370751 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.370720    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.370936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.370817    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzwsw\" (UniqueName: \"kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.370936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.370844    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.370936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.370873    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.370936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.370914    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.371163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.370962    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.371344 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.371322    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.372912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.372887    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.373029 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.372985    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.373558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.373539    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.375270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.375251    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.428354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.428273    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzwsw\" (UniqueName: \"kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw\") pod \"simulator-w5-e210ef-kserve-844bbc78c9-ztjv4\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") " pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.460087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.460045    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:36.837200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:36.837125    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"]
Nov 04 01:12:36.840251 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:12:36.840222    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabfe33ca_9ba1_42a1_a18f_8ccdcea7c8ea.slice/crio-926d472b90c6a1e6a53fb97b97444e36aa1551368fce0ce1f8b51bdbc5b1d393 WatchSource:0}: Error finding container 926d472b90c6a1e6a53fb97b97444e36aa1551368fce0ce1f8b51bdbc5b1d393: Status 404 returned error can't find the container with id 926d472b90c6a1e6a53fb97b97444e36aa1551368fce0ce1f8b51bdbc5b1d393
Nov 04 01:12:37.326512 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:37.326476    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" event={"ID":"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea","Type":"ContainerStarted","Data":"8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416"}
Nov 04 01:12:37.326924 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:37.326521    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" event={"ID":"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea","Type":"ContainerStarted","Data":"926d472b90c6a1e6a53fb97b97444e36aa1551368fce0ce1f8b51bdbc5b1d393"}
Nov 04 01:12:37.443079 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:37.443020    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podStartSLOduration=1.4430038920000001 podStartE2EDuration="1.443003892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:37.441171574 +0000 UTC m=+926.103311629" watchObservedRunningTime="2026-11-04 01:12:37.443003892 +0000 UTC m=+926.105143953"
Nov 04 01:12:37.461221 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:37.461166    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:39.469187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:39.469151    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:44.393972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:44.393936    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:44.726335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:44.726254    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:46.460751 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:46.460715    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:46.473450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:46.473425    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:47.385126 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:47.385094    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:12:49.263871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:49.263834    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:51.896715 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:12:51.896652    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:12:54.169501 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:54.169419    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:56.666037 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.666002    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"]
Nov 04 01:12:56.670783 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.670762    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.723849 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.723816    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"]
Nov 04 01:12:56.767428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.767391    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vrrr\" (UniqueName: \"kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.767428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.767435    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.767649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.767504    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.767649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.767601    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.767649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.767632    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.767840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.767652    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.771991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.771969    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e65b23\"/\"default-dockercfg-fjq9w\""
Nov 04 01:12:56.772114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.772042    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e65b23\"/\"openshift-service-ca.crt\""
Nov 04 01:12:56.772114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.772109    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e65b23\"/\"simulator-w4-e65b23-kserve-self-signed-certs\""
Nov 04 01:12:56.773535 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.773521    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e65b23\"/\"kube-root-ca.crt\""
Nov 04 01:12:56.785006 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.784968    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:56.868860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.868825    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vrrr\" (UniqueName: \"kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.868860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.868868    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.868909    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.868932    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.868954    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.869024    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.869403    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.869433    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.869587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.869501    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.871264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.871239    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.872197 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.872179    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.926474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.926397    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vrrr\" (UniqueName: \"kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr\") pod \"simulator-w4-e65b23-kserve-6666544c66-6s6zs\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") " pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:56.983097 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:56.983064    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:57.365549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:57.365466    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"]
Nov 04 01:12:57.368525 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:12:57.368496    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fccb086_7049_42d2_818e_2c16e8492cee.slice/crio-37430d26335498d92219eaebc3545a94c2c5b2818fe26bc9578cd1a182e09b06 WatchSource:0}: Error finding container 37430d26335498d92219eaebc3545a94c2c5b2818fe26bc9578cd1a182e09b06: Status 404 returned error can't find the container with id 37430d26335498d92219eaebc3545a94c2c5b2818fe26bc9578cd1a182e09b06
Nov 04 01:12:57.421648 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:57.421619    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" event={"ID":"2fccb086-7049-42d2-818e-2c16e8492cee","Type":"ContainerStarted","Data":"37430d26335498d92219eaebc3545a94c2c5b2818fe26bc9578cd1a182e09b06"}
Nov 04 01:12:58.427997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:58.427956    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" event={"ID":"2fccb086-7049-42d2-818e-2c16e8492cee","Type":"ContainerStarted","Data":"220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7"}
Nov 04 01:12:58.543896 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:58.543836    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podStartSLOduration=2.543821 podStartE2EDuration="2.543821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:12:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:12:58.542606075 +0000 UTC m=+947.204746141" watchObservedRunningTime="2026-11-04 01:12:58.543821 +0000 UTC m=+947.205961060"
Nov 04 01:12:58.983963 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:58.983927    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:12:59.141171 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:59.141123    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:12:59.845571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:12:59.845540    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:03.984871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:03.984830    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:06.983652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:06.983619    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:13:06.996388 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:06.996364    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:13:07.484088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:07.484059    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:13:08.877428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:08.877374    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:13.672044 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:13.672005    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:13.832428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:13.832388    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:17.238104 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:17.238065    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:18.806967 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:18.806934    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:21.896308 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:13:21.896267    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:13:23.672933 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:23.672899    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:28.575518 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:28.575478    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:30.774979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.774946    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"]
Nov 04 01:13:30.778785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.778767    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.805057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.805030    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.805210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.805067    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.805210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.805191    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.805321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.805229    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.805321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.805307    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7m45\" (UniqueName: \"kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.805411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.805351    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.839799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.839757    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"]
Nov 04 01:13:30.880583 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.880543    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-678aaf\"/\"premium-w6-678aaf-kserve-self-signed-certs\""
Nov 04 01:13:30.892835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.892801    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:30.906118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906090    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7m45\" (UniqueName: \"kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906131    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906199    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906214    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906405 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906263    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906405 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906297    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906543    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906628 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906608    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.906749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.906724    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.908475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.908452    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.908758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.908740    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:30.963318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:30.963283    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7m45\" (UniqueName: \"kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45\") pod \"premium-w6-678aaf-kserve-55d99bc649-p29hz\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") " pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:31.090274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:31.090180    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:31.475111 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:13:31.475083    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod814094e6_b400_482c_b461_3957063e72e3.slice/crio-f556dd8f54a85d5bc0050d031169efcb06d83875806f75477e3c9ff3377e210a WatchSource:0}: Error finding container f556dd8f54a85d5bc0050d031169efcb06d83875806f75477e3c9ff3377e210a: Status 404 returned error can't find the container with id f556dd8f54a85d5bc0050d031169efcb06d83875806f75477e3c9ff3377e210a
Nov 04 01:13:31.475336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:31.475311    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"]
Nov 04 01:13:31.598012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:31.597970    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" event={"ID":"814094e6-b400-482c-b461-3957063e72e3","Type":"ContainerStarted","Data":"033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128"}
Nov 04 01:13:31.598012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:31.598018    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" event={"ID":"814094e6-b400-482c-b461-3957063e72e3","Type":"ContainerStarted","Data":"f556dd8f54a85d5bc0050d031169efcb06d83875806f75477e3c9ff3377e210a"}
Nov 04 01:13:31.712235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:31.712181    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podStartSLOduration=1.712166706 podStartE2EDuration="1.712166706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:13:31.711175377 +0000 UTC m=+980.373315438" watchObservedRunningTime="2026-11-04 01:13:31.712166706 +0000 UTC m=+980.374306767"
Nov 04 01:13:32.091285 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:32.091249    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:33.386360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:33.386323    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:38.301887 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:38.301851    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:41.091154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:41.091114    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:41.104461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:41.104435    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:41.658565 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:41.658536    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:13:43.278846 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:43.278802    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:48.233080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:48.233047    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:51.897563 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:13:51.897525    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:13:53.098817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:53.098776    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:55.769857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:55.769796    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:13:58.103534 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:13:58.103482    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:00.192577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:00.192539    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:02.962920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:02.962878    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:07.974978 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:07.974940    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:12.982117 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:12.982080    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:16.648075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:16.648034    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:17.874276 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:17.874246    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:21.206874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:21.206838    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:21.896695 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:14:21.896650    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:14:22.910049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:22.910010    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:27.898807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:27.898751    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:28.582176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.582138    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"]
Nov 04 01:14:28.586302 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.586285    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.590730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.590705    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.590856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.590773    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.590856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.590842    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x6lf\" (UniqueName: \"kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.590933 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.590873    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.590933 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.590897    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.591041 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.591017    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.641184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.641151    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"]
Nov 04 01:14:28.688775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.688749    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e65b23\"/\"premium-w4-e65b23-kserve-self-signed-certs\""
Nov 04 01:14:28.692292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692271    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692312    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x6lf\" (UniqueName: \"kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692345    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692361    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692389    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692422    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692717    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692780    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.692861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.692808    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.694769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.694742    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.694896 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.694881    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.706679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.706644    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:28.749749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.749722    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x6lf\" (UniqueName: \"kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf\") pod \"premium-w4-e65b23-kserve-59f874f59f-wknwk\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") " pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:28.898997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:28.898970    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:29.276757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:29.276678    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"]
Nov 04 01:14:29.279958 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:14:29.279932    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc30d109b_c451_4de5_90e4_9a1a670762d5.slice/crio-f66bcfef057acec90869f0ad4e06881aa12751c28189a2cbdbb7ab8d90c7fedc WatchSource:0}: Error finding container f66bcfef057acec90869f0ad4e06881aa12751c28189a2cbdbb7ab8d90c7fedc: Status 404 returned error can't find the container with id f66bcfef057acec90869f0ad4e06881aa12751c28189a2cbdbb7ab8d90c7fedc
Nov 04 01:14:29.878080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:29.878035    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" event={"ID":"c30d109b-c451-4de5-90e4-9a1a670762d5","Type":"ContainerStarted","Data":"59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8"}
Nov 04 01:14:29.878258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:29.878089    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" event={"ID":"c30d109b-c451-4de5-90e4-9a1a670762d5","Type":"ContainerStarted","Data":"f66bcfef057acec90869f0ad4e06881aa12751c28189a2cbdbb7ab8d90c7fedc"}
Nov 04 01:14:29.899354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:29.899326    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:29.994216 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:29.994162    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podStartSLOduration=1.994147358 podStartE2EDuration="1.994147358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:14:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:14:29.992059024 +0000 UTC m=+1038.654199109" watchObservedRunningTime="2026-11-04 01:14:29.994147358 +0000 UTC m=+1038.656287419"
Nov 04 01:14:32.867803 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:32.867769    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:37.766978 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:37.766942    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:38.265706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:38.265645    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:38.899430 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:38.899393    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:38.913652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:38.913629    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:38.933760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:38.933737    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:14:42.789088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:42.789049    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:47.667521 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:47.667475    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:51.896893 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:14:51.896860    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:14:52.678089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:52.678039    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:14:57.608488 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:14:57.608430    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:00.193462 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.193420    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"]
Nov 04 01:15:00.197378 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.197356    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.249616 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.249578    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qb9zm\""
Nov 04 01:15:00.301094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.301055    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.301287 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.301235    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gdr\" (UniqueName: \"kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr\") pod \"maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.388530 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.387998    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"]
Nov 04 01:15:00.402349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.402321    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.402514 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.402452    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gdr\" (UniqueName: \"kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr\") pod \"maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.403015 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.402996    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.459782 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.459703    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gdr\" (UniqueName: \"kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr\") pod \"maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.511654 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.511625    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:00.533135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.533094    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:00.886051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:00.886013    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"]
Nov 04 01:15:00.890230 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:15:00.890191    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5022498_6e04_49f8_b507_8e1d75b356c1.slice/crio-ba8486e357f9ca9c3d37995f79bf10e348125025fda8cac1fe3f596f8f2ff64f WatchSource:0}: Error finding container ba8486e357f9ca9c3d37995f79bf10e348125025fda8cac1fe3f596f8f2ff64f: Status 404 returned error can't find the container with id ba8486e357f9ca9c3d37995f79bf10e348125025fda8cac1fe3f596f8f2ff64f
Nov 04 01:15:01.033993 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:01.033958    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" event={"ID":"e5022498-6e04-49f8-b507-8e1d75b356c1","Type":"ContainerStarted","Data":"ba8486e357f9ca9c3d37995f79bf10e348125025fda8cac1fe3f596f8f2ff64f"}
Nov 04 01:15:02.797439 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:02.797400    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:06.059858 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:06.059766    4434 generic.go:356] "Generic (PLEG): container finished" podID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerID="cf8eaacbc14f609321d314112d584cf5a4763474130bc9dbb20775cc161457d3" exitCode=60
Nov 04 01:15:06.060252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:06.059852    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" event={"ID":"e5022498-6e04-49f8-b507-8e1d75b356c1","Type":"ContainerDied","Data":"cf8eaacbc14f609321d314112d584cf5a4763474130bc9dbb20775cc161457d3"}
Nov 04 01:15:06.060252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:06.060137    4434 scope.go:122] "RemoveContainer" containerID="cf8eaacbc14f609321d314112d584cf5a4763474130bc9dbb20775cc161457d3"
Nov 04 01:15:07.066366 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:07.066329    4434 generic.go:356] "Generic (PLEG): container finished" podID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerID="a6f18bfec387be0620a788981a1b135cb8db9d7daa40cc5b8292cb7fae964284" exitCode=60
Nov 04 01:15:07.066823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:07.066417    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" event={"ID":"e5022498-6e04-49f8-b507-8e1d75b356c1","Type":"ContainerDied","Data":"a6f18bfec387be0620a788981a1b135cb8db9d7daa40cc5b8292cb7fae964284"}
Nov 04 01:15:07.066823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:07.066485    4434 scope.go:122] "RemoveContainer" containerID="cf8eaacbc14f609321d314112d584cf5a4763474130bc9dbb20775cc161457d3"
Nov 04 01:15:07.066823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:07.066736    4434 scope.go:122] "RemoveContainer" containerID="a6f18bfec387be0620a788981a1b135cb8db9d7daa40cc5b8292cb7fae964284"
Nov 04 01:15:07.067014 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:15:07.066996    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2_odh-ai-gateway-infra(e5022498-6e04-49f8-b507-8e1d75b356c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1"
Nov 04 01:15:07.699716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:07.699655    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:08.073431 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:08.073354    4434 scope.go:122] "RemoveContainer" containerID="a6f18bfec387be0620a788981a1b135cb8db9d7daa40cc5b8292cb7fae964284"
Nov 04 01:15:08.073849 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:15:08.073597    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2_odh-ai-gateway-infra(e5022498-6e04-49f8-b507-8e1d75b356c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1"
Nov 04 01:15:10.253226 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:10.253189    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"]
Nov 04 01:15:10.253715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:10.253401    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy" containerID="cri-o://810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22" gracePeriod=30
Nov 04 01:15:12.594944 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:12.594902    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:15.497751 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.497725    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:15:15.568312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568234    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568282    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568361    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568396    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568423    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568460    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568493    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568525    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568537    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:15:15.568874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568552    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token\") pod \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\" (UID: \"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7\") "
Nov 04 01:15:15.568874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568806    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:15:15.568874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568856    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:15:15.569089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568869    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:15:15.569089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.568896    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:15:15.569185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.569133    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.569185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.569159    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.569185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.569173    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.569745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.569189    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.569745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.569202    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.570949 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.570921    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:15:15.571050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.570964    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:15:15.571050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.570990    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "kube-api-access-dpbmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:15:15.571122 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.571045    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo" pod "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" (UID: "0b9f3f39-fdf6-41c9-943c-0f48fae93fe7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:15:15.670352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.670320    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpbmq\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-kube-api-access-dpbmq\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.670352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.670350    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.670352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.670361    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.670581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.670370    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:15.931484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.931444    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"]
Nov 04 01:15:15.931958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.931941    4434 cpu_manager.go:414] "containerMap: removing container" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy"
Nov 04 01:15:15.932016 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.931960    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy"
Nov 04 01:15:15.932062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.932043    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerName="istio-proxy"
Nov 04 01:15:15.935489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.935468    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:15.973741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.973711    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:15.973887 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.973866    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:15.973937 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.973927    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnxzr\" (UniqueName: \"kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:15.988255 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.988229    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:15:15.990256 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:15.990229    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"]
Nov 04 01:15:16.074728 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.074692    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.074894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.074752    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnxzr\" (UniqueName: \"kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.074894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.074778    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.075427 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.075396    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.077175 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.077154    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.111964 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.111932    4434 generic.go:356] "Generic (PLEG): container finished" podID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" containerID="810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22" exitCode=0
Nov 04 01:15:16.112106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.111977    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"
Nov 04 01:15:16.112106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.111988    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" event={"ID":"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7","Type":"ContainerDied","Data":"810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22"}
Nov 04 01:15:16.112106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.112034    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc" event={"ID":"0b9f3f39-fdf6-41c9-943c-0f48fae93fe7","Type":"ContainerDied","Data":"30a82158e6f03b1205f8256de1ad8d1c3a0598d35e392a457265304aaf4e70ef"}
Nov 04 01:15:16.112106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.112062    4434 scope.go:122] "RemoveContainer" containerID="810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22"
Nov 04 01:15:16.121484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.121469    4434 scope.go:122] "RemoveContainer" containerID="810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22"
Nov 04 01:15:16.121871 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:15:16.121849    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22\": container with ID starting with 810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22 not found: ID does not exist" containerID="810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22"
Nov 04 01:15:16.121945 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.121879    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22"} err="failed to get container status \"810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22\": rpc error: code = NotFound desc = could not find container \"810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22\": container with ID starting with 810a1adac1ebed2ff18381a18e9caee0ba0df60e42dae07c2e25f0d81684ca22 not found: ID does not exist"
Nov 04 01:15:16.132140 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.132120    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnxzr\" (UniqueName: \"kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr\") pod \"maas-api-revoke-sub-0801cec9-sv57f\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.248379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.248287    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:16.279708 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.279644    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"]
Nov 04 01:15:16.283506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.283473    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-hcxkc"]
Nov 04 01:15:16.628955 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:16.628917    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"]
Nov 04 01:15:16.633925 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:15:16.633892    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf434e13b_c5a4_4d82_a067_278aaa95cc4b.slice/crio-85199ffba629f52e6d40e92b263feadcc930a183cd6ccac9be479afacb49b487 WatchSource:0}: Error finding container 85199ffba629f52e6d40e92b263feadcc930a183cd6ccac9be479afacb49b487: Status 404 returned error can't find the container with id 85199ffba629f52e6d40e92b263feadcc930a183cd6ccac9be479afacb49b487
Nov 04 01:15:17.118316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:17.118277    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f" event={"ID":"f434e13b-c5a4-4d82-a067-278aaa95cc4b","Type":"ContainerStarted","Data":"ddfd8e1ec00674808840e2d8742c35c463db986a27885a409e2c6fe50f3ea9e7"}
Nov 04 01:15:17.118525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:17.118336    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f" event={"ID":"f434e13b-c5a4-4d82-a067-278aaa95cc4b","Type":"ContainerStarted","Data":"85199ffba629f52e6d40e92b263feadcc930a183cd6ccac9be479afacb49b487"}
Nov 04 01:15:17.234979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:17.234888    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f" podStartSLOduration=2.2348740400000002 podStartE2EDuration="2.23487404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:17.233498226 +0000 UTC m=+1085.895638286" watchObservedRunningTime="2026-11-04 01:15:17.23487404 +0000 UTC m=+1085.897014101"
Nov 04 01:15:17.475912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:17.475876    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b9f3f39-fdf6-41c9-943c-0f48fae93fe7" path="/var/lib/kubelet/pods/0b9f3f39-fdf6-41c9-943c-0f48fae93fe7/volumes"
Nov 04 01:15:17.723432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:17.723393    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:18.125765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:18.125728    4434 generic.go:356] "Generic (PLEG): container finished" podID="f434e13b-c5a4-4d82-a067-278aaa95cc4b" containerID="ddfd8e1ec00674808840e2d8742c35c463db986a27885a409e2c6fe50f3ea9e7" exitCode=0
Nov 04 01:15:18.125929 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:18.125808    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f" event={"ID":"f434e13b-c5a4-4d82-a067-278aaa95cc4b","Type":"ContainerDied","Data":"ddfd8e1ec00674808840e2d8742c35c463db986a27885a409e2c6fe50f3ea9e7"}
Nov 04 01:15:19.271351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.271324    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:19.411864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.411775    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca\") pod \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") "
Nov 04 01:15:19.412025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.411933    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca\") pod \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") "
Nov 04 01:15:19.412025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.411984    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr\" (UniqueName: \"kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr\") pod \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\" (UID: \"f434e13b-c5a4-4d82-a067-278aaa95cc4b\") "
Nov 04 01:15:19.412211 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.412187    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca" pod "f434e13b-c5a4-4d82-a067-278aaa95cc4b" (UID: "f434e13b-c5a4-4d82-a067-278aaa95cc4b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:15:19.414033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.414004    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr" pod "f434e13b-c5a4-4d82-a067-278aaa95cc4b" (UID: "f434e13b-c5a4-4d82-a067-278aaa95cc4b"). InnerVolumeSpecName "kube-api-access-dnxzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:15:19.414107 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.414072    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca" pod "f434e13b-c5a4-4d82-a067-278aaa95cc4b" (UID: "f434e13b-c5a4-4d82-a067-278aaa95cc4b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:15:19.474338 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.474304    4434 scope.go:122] "RemoveContainer" containerID="a6f18bfec387be0620a788981a1b135cb8db9d7daa40cc5b8292cb7fae964284"
Nov 04 01:15:19.513338 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.513308    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-openshift-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:19.513338 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.513335    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f434e13b-c5a4-4d82-a067-278aaa95cc4b-maas-api-xks-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:19.513508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:19.513346    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnxzr\" (UniqueName: \"kubernetes.io/projected/f434e13b-c5a4-4d82-a067-278aaa95cc4b-kube-api-access-dnxzr\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:20.137306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:20.137274    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"
Nov 04 01:15:20.137306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:20.137283    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f" event={"ID":"f434e13b-c5a4-4d82-a067-278aaa95cc4b","Type":"ContainerDied","Data":"85199ffba629f52e6d40e92b263feadcc930a183cd6ccac9be479afacb49b487"}
Nov 04 01:15:20.137526 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:20.137337    4434 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85199ffba629f52e6d40e92b263feadcc930a183cd6ccac9be479afacb49b487"
Nov 04 01:15:20.139420 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:20.139396    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" event={"ID":"e5022498-6e04-49f8-b507-8e1d75b356c1","Type":"ContainerStarted","Data":"a6e6a9439fb0a4378bc15558c42cdb7eed4b43666c707403acb53e39d2e6a7b7"}
Nov 04 01:15:20.255243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:20.255186    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" podStartSLOduration=16.209718084 podStartE2EDuration="20.255172074s" totalImagesPullingTime="4.04545399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:20.254138414 +0000 UTC m=+1088.916278472" watchObservedRunningTime="2026-11-04 01:15:20.255172074 +0000 UTC m=+1088.917312177"
Nov 04 01:15:21.146460 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:21.146427    4434 generic.go:356] "Generic (PLEG): container finished" podID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerID="a6e6a9439fb0a4378bc15558c42cdb7eed4b43666c707403acb53e39d2e6a7b7" exitCode=60
Nov 04 01:15:21.146917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:21.146499    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" event={"ID":"e5022498-6e04-49f8-b507-8e1d75b356c1","Type":"ContainerDied","Data":"a6e6a9439fb0a4378bc15558c42cdb7eed4b43666c707403acb53e39d2e6a7b7"}
Nov 04 01:15:21.146917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:21.146560    4434 scope.go:122] "RemoveContainer" containerID="a6f18bfec387be0620a788981a1b135cb8db9d7daa40cc5b8292cb7fae964284"
Nov 04 01:15:21.146917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:21.146877    4434 scope.go:122] "RemoveContainer" containerID="a6e6a9439fb0a4378bc15558c42cdb7eed4b43666c707403acb53e39d2e6a7b7"
Nov 04 01:15:21.147155 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:15:21.147120    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2_odh-ai-gateway-infra(e5022498-6e04-49f8-b507-8e1d75b356c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1"
Nov 04 01:15:21.271963 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:21.271933    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"]
Nov 04 01:15:21.896535 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:15:21.896501    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:15:22.281006 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.280979    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:22.342912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.342886    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca\") pod \"e5022498-6e04-49f8-b507-8e1d75b356c1\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") "
Nov 04 01:15:22.343057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.343002    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr\" (UniqueName: \"kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr\") pod \"e5022498-6e04-49f8-b507-8e1d75b356c1\" (UID: \"e5022498-6e04-49f8-b507-8e1d75b356c1\") "
Nov 04 01:15:22.343227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.343204    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca" pod "e5022498-6e04-49f8-b507-8e1d75b356c1" (UID: "e5022498-6e04-49f8-b507-8e1d75b356c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:15:22.344994 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.344975    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr" pod "e5022498-6e04-49f8-b507-8e1d75b356c1" (UID: "e5022498-6e04-49f8-b507-8e1d75b356c1"). InnerVolumeSpecName "kube-api-access-v9gdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:15:22.444067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.443991    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5022498-6e04-49f8-b507-8e1d75b356c1-maas-api-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:22.444067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.444015    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9gdr\" (UniqueName: \"kubernetes.io/projected/e5022498-6e04-49f8-b507-8e1d75b356c1-kube-api-access-v9gdr\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:15:22.637116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:22.637081    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:23.158513 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:23.158479    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"
Nov 04 01:15:23.158693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:23.158476    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2" event={"ID":"e5022498-6e04-49f8-b507-8e1d75b356c1","Type":"ContainerDied","Data":"ba8486e357f9ca9c3d37995f79bf10e348125025fda8cac1fe3f596f8f2ff64f"}
Nov 04 01:15:23.158693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:23.158640    4434 scope.go:122] "RemoveContainer" containerID="a6e6a9439fb0a4378bc15558c42cdb7eed4b43666c707403acb53e39d2e6a7b7"
Nov 04 01:15:23.323536 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:23.323500    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"]
Nov 04 01:15:23.373732 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:23.373693    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-e210ef-29895915-xxbv2"]
Nov 04 01:15:23.474369 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:23.474280    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" path="/var/lib/kubelet/pods/e5022498-6e04-49f8-b507-8e1d75b356c1/volumes"
Nov 04 01:15:27.877250 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:27.877216    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:32.767221 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:32.767182    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:37.676757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:37.676703    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:40.273132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273102    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"]
Nov 04 01:15:40.273561 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273534    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.273561 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273547    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.273561 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273559    4434 cpu_manager.go:414] "containerMap: removing container" podUID="f434e13b-c5a4-4d82-a067-278aaa95cc4b" containerName="revoke-keys"
Nov 04 01:15:40.273561 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273563    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f434e13b-c5a4-4d82-a067-278aaa95cc4b" containerName="revoke-keys"
Nov 04 01:15:40.273806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273573    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.273806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273578    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.273806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273683    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.273806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273694    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.273806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273700    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="f434e13b-c5a4-4d82-a067-278aaa95cc4b" containerName="revoke-keys"
Nov 04 01:15:40.273806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.273707    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:40.278345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.278315    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.325900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.325870    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqjt5\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.325909    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326017    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326058    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326084    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326161    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326190    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326276 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326240    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.326276 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.326261    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.426979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.426940    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.426991    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427027    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427063    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427098    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427154    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427183    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427223    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqjt5\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427263    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427472    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427529    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427635    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.427965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.427943    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.428347 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.428321    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.429385 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.429361    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.429794 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.429772    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.471760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.471730    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"]
Nov 04 01:15:40.487647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.487617    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqjt5\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.487647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.487627    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-m9nls\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:40.590887 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:40.590809    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:41.016610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.016584    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"]
Nov 04 01:15:41.019153 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:15:41.019118    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd4e2be_7854_48c2_9cf3_09425a736b43.slice/crio-0c4d98f16824d6b68a7312cecc7223fa5d0369b759f1351ea91caa8a01e10a2c WatchSource:0}: Error finding container 0c4d98f16824d6b68a7312cecc7223fa5d0369b759f1351ea91caa8a01e10a2c: Status 404 returned error can't find the container with id 0c4d98f16824d6b68a7312cecc7223fa5d0369b759f1351ea91caa8a01e10a2c
Nov 04 01:15:41.021357 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.021321    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:41.021476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.021399    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:41.021476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.021447    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:41.251320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.251273    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" event={"ID":"3bd4e2be-7854-48c2-9cf3-09425a736b43","Type":"ContainerStarted","Data":"5e2be7fe2673c4a320df537a6f446b4f4c1b9ff75bb0e9a7e0c9244a74721dcb"}
Nov 04 01:15:41.251495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.251346    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" event={"ID":"3bd4e2be-7854-48c2-9cf3-09425a736b43","Type":"ContainerStarted","Data":"0c4d98f16824d6b68a7312cecc7223fa5d0369b759f1351ea91caa8a01e10a2c"}
Nov 04 01:15:41.369949 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.369860    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podStartSLOduration=1.36984755 podStartE2EDuration="1.36984755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:41.368195978 +0000 UTC m=+1110.030336044" watchObservedRunningTime="2026-11-04 01:15:41.36984755 +0000 UTC m=+1110.031987613"
Nov 04 01:15:41.590996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:41.590960    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:42.592937 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:42.592897    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-m9nls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Nov 04 01:15:42.593364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:42.592964    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Nov 04 01:15:42.633050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:42.633022    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:43.591473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:43.591439    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-m9nls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Nov 04 01:15:43.591626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:43.591488    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Nov 04 01:15:44.591248 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:44.591211    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-m9nls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Nov 04 01:15:44.591768 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:44.591273    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Nov 04 01:15:45.591292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:45.591257    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-m9nls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Nov 04 01:15:45.591829 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:45.591317    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Nov 04 01:15:46.488850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.485329    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"]
Nov 04 01:15:46.488850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.486742    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:46.488850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.486764    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5022498-6e04-49f8-b507-8e1d75b356c1" containerName="cleanup"
Nov 04 01:15:46.491766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.491745    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511584 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511555    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511596    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511773 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511711    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511773    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511793    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxbsv\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511814    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511847    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511864    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.511911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.511890    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.544874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.544845    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"]
Nov 04 01:15:46.592056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.592009    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-m9nls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Nov 04 01:15:46.592501 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.592082    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Nov 04 01:15:46.592784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.592755    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-2fd27d-openshift-default-dockercfg-bkzjb\""
Nov 04 01:15:46.613301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613268    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613308    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613444    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613587    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613625    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613643    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613707    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613709    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613740    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613774    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxbsv\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.613814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613797    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.614163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.613910    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.614163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.614026    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.614403 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.614382    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.615884 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.615866    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.616086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.616070    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.691017 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.690981    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.691161 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.691071    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxbsv\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:46.805795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:46.805711    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:47.233605 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.233581    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"]
Nov 04 01:15:47.237685 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:15:47.237630    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf393871e_3490_4f66_86c1_dff15707249e.slice/crio-5de45195575b29ddce93e33d1457770624eab428205fdb7873f7f8059e38abdf WatchSource:0}: Error finding container 5de45195575b29ddce93e33d1457770624eab428205fdb7873f7f8059e38abdf: Status 404 returned error can't find the container with id 5de45195575b29ddce93e33d1457770624eab428205fdb7873f7f8059e38abdf
Nov 04 01:15:47.239575 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.239542    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:47.239651 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.239614    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:47.239725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.239653    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:47.308222 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.308193    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" event={"ID":"f393871e-3490-4f66-86c1-dff15707249e","Type":"ContainerStarted","Data":"4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d"}
Nov 04 01:15:47.308332 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.308242    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" event={"ID":"f393871e-3490-4f66-86c1-dff15707249e","Type":"ContainerStarted","Data":"5de45195575b29ddce93e33d1457770624eab428205fdb7873f7f8059e38abdf"}
Nov 04 01:15:47.429469 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.429411    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podStartSLOduration=1.429391775 podStartE2EDuration="1.429391775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:47.426100811 +0000 UTC m=+1116.088240874" watchObservedRunningTime="2026-11-04 01:15:47.429391775 +0000 UTC m=+1116.091531838"
Nov 04 01:15:47.591749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.591652    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-m9nls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Nov 04 01:15:47.591749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.591734    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Nov 04 01:15:47.667644 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.667609    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:47.806444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:47.806407    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:48.808854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:48.808817    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:15:48.809351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:48.809322    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 01:15:49.544090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:49.544057    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:49.544742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:49.544698    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:49.545506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:49.545489    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:15:49.806468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:49.806379    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:15:49.806468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:49.806437    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 01:15:50.806895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:50.806862    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:15:50.807293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:50.806916    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 01:15:51.806035 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:51.806002    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:15:51.806231 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:51.806051    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 01:15:51.915625 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:15:51.915585    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:15:52.654896 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:52.654853    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:52.807021 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:52.806984    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Nov 04 01:15:52.807224 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:52.807049    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Nov 04 01:15:53.888783 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:53.888702    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:53.889241 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:53.889144    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:53.890037 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:53.890015    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:15:55.261070 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.261039    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"]
Nov 04 01:15:55.263905 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.263886    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.368612 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.368582    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"]
Nov 04 01:15:55.425908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.425870    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwgmj\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.425924    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426042    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426179 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426150    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426179 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426176    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426260 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426199    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426260 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426233    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426260 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426253    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.426470 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.426453    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527518    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwgmj\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527566    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527592    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527630    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527647    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527702    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527745    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527774    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.527878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527832    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.528187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.527990    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.528187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.528078    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.528187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.528134    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.528293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.528184    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.528718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.528700    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.529985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.529960    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.530181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.530164    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.584768 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.584739    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.585225 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.585201    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwgmj\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj\") pod \"maas-default-gateway-openshift-default-694db55c87-lxc9v\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:55.876363 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:55.876329    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:56.301729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:56.301612    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"]
Nov 04 01:15:56.306185 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:15:56.306153    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1889a421_b757_4bad_8b15_f8ff87ae8e9e.slice/crio-1fe561679c5f697c418d33faac4fc39e34100b1a73502da6043141632f339d8b WatchSource:0}: Error finding container 1fe561679c5f697c418d33faac4fc39e34100b1a73502da6043141632f339d8b: Status 404 returned error can't find the container with id 1fe561679c5f697c418d33faac4fc39e34100b1a73502da6043141632f339d8b
Nov 04 01:15:56.309415 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:56.309139    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:56.310642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:56.309292    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:56.310642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:56.309652    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:15:56.356994 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:56.356959    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" event={"ID":"1889a421-b757-4bad-8b15-f8ff87ae8e9e","Type":"ContainerStarted","Data":"1fe561679c5f697c418d33faac4fc39e34100b1a73502da6043141632f339d8b"}
Nov 04 01:15:57.369686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:57.368891    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" event={"ID":"1889a421-b757-4bad-8b15-f8ff87ae8e9e","Type":"ContainerStarted","Data":"838b109e6bfb62ad262710e0c06ea33ca302b994584b3ee04cc2b9124eeeca02"}
Nov 04 01:15:57.547054 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:57.547000    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podStartSLOduration=2.546986799 podStartE2EDuration="2.546986799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:15:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:15:57.544975246 +0000 UTC m=+1126.207115320" watchObservedRunningTime="2026-11-04 01:15:57.546986799 +0000 UTC m=+1126.209126860"
Nov 04 01:15:57.880689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:57.877843    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:15:57.880689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:57.880466    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:15:57.880689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:57.880524    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:15:57.898698 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:57.898649    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:58.254188 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:58.254105    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:15:58.879690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:58.879004    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:15:58.879690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:58.879063    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:15:59.881934 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:59.881898    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:15:59.882492 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:15:59.882462    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:16:00.881692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:00.878054    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:16:00.881692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:00.878118    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:16:01.019352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:01.019313    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:01.882690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:01.877087    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:16:01.882690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:01.877188    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:16:02.877344 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:02.877301    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:16:02.877756 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:02.877386    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:16:03.294103 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:03.293975    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:03.764109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:03.764064    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:03.877651 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:03.877608    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 04 01:16:03.878241 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:03.877709    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 04 01:16:05.917822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:05.917780    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-lxc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:16:05.918356 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:05.917838    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:16:06.889696 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:06.889642    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:16:06.890083 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:06.890061    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:16:06.891289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:06.891265    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:16:08.275827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:08.275792    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:13.223909 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:13.223871    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:18.108497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:18.108456    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:21.897187 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:16:21.897146    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:16:23.217562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:23.217530    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:28.176817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:28.176776    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:32.159493 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.159460    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"]
Nov 04 01:16:32.162972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.162953    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.217425 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.217390    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-e993d7-openshift-default-dockercfg-fr6p4\""
Nov 04 01:16:32.270033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.269999    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"]
Nov 04 01:16:32.339132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339103    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339143    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339215    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339260    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339325    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339419    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339478 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339465    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339513 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339491    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqnx\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.339546 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.339531    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440320    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440356    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqnx\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440379    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440437    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440463    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440601    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440692    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440720    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440803    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440808    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.440965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.440810    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.441133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.441009    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.441243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.441222    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.441496 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.441478    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.442793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.442769    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.443005 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.442990    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.497684 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.497648    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.497968 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.497952    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqnx\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:32.775691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:32.775580    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:33.081184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.081068    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:33.221553 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.221444    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"]
Nov 04 01:16:33.224269 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:16:33.224243    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc479376a_8b56_48e9_8104_46518e09c0a5.slice/crio-ae644bae3d34e17cb279b7820f4aeb8f6b9d9cb9b35cfa681b9517f2d3f5636d WatchSource:0}: Error finding container ae644bae3d34e17cb279b7820f4aeb8f6b9d9cb9b35cfa681b9517f2d3f5636d: Status 404 returned error can't find the container with id ae644bae3d34e17cb279b7820f4aeb8f6b9d9cb9b35cfa681b9517f2d3f5636d
Nov 04 01:16:33.227011 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.226981    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:33.227124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.227055    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:33.227124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.227098    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:33.581891 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.581852    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" event={"ID":"c479376a-8b56-48e9-8104-46518e09c0a5","Type":"ContainerStarted","Data":"f04995bf6b006b37860a1eb0329640f6cd5434a857e69a0f9f2ee0346394f428"}
Nov 04 01:16:33.582064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.581908    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" event={"ID":"c479376a-8b56-48e9-8104-46518e09c0a5","Type":"ContainerStarted","Data":"ae644bae3d34e17cb279b7820f4aeb8f6b9d9cb9b35cfa681b9517f2d3f5636d"}
Nov 04 01:16:33.699978 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.699928    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podStartSLOduration=1.6999137640000002 podStartE2EDuration="1.699913764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:33.698810746 +0000 UTC m=+1162.360950806" watchObservedRunningTime="2026-11-04 01:16:33.699913764 +0000 UTC m=+1162.362053824"
Nov 04 01:16:33.775979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:33.775929    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:34.787816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:34.787774    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 04 01:16:34.788555 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:34.788520    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 04 01:16:35.783091 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:35.783049    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 04 01:16:35.783388 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:35.783362    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 04 01:16:36.786692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:36.779032    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 04 01:16:36.786692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:36.779094    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 04 01:16:37.782116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:37.782074    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 04 01:16:37.782405 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:37.782372    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 04 01:16:38.090706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:38.090599    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:38.777046 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:38.777007    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 04 01:16:38.778292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:38.778261    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 04 01:16:39.981573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:39.981538    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:39.982997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:39.982969    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:39.983487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:39.983471    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:16:43.001623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:43.001588    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:46.607536 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.607503    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"]
Nov 04 01:16:46.611843 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.611826    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.664874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.664852    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-678aaf-openshift-default-dockercfg-88gm8\""
Nov 04 01:16:46.717635 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.717605    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"]
Nov 04 01:16:46.720833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.720807    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.720965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.720840    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.720965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.720872    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.720965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.720939    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.721139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.721040    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.721178 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.721136    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gt77\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.721228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.721203    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.721270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.721230    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.721270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.721245    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.821959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.821927    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.821963    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822002    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822037    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gt77\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822107    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822144    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822165    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822211    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822238    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822520    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822680 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822568    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822680 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822617    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822784    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.822982 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.822960    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.824409 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.824386    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.824551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.824535    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.878801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.878774    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.878953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.878939    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gt77\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:46.924685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:46.924635    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:47.359689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.359636    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"]
Nov 04 01:16:47.361637 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:16:47.361610    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4bffc01_0d12_438e_aaec_7873e04b837d.slice/crio-6fee771903ed914b09b0c42ba05decb18b2d1af1a5d2756906342b322aa910f6 WatchSource:0}: Error finding container 6fee771903ed914b09b0c42ba05decb18b2d1af1a5d2756906342b322aa910f6: Status 404 returned error can't find the container with id 6fee771903ed914b09b0c42ba05decb18b2d1af1a5d2756906342b322aa910f6
Nov 04 01:16:47.364063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.364024    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:47.364138 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.364095    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:47.364138 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.364123    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:47.660701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.660641    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" event={"ID":"c4bffc01-0d12-438e-aaec-7873e04b837d","Type":"ContainerStarted","Data":"61e94d4eba1826b5fc0fc28846e2c7bcf947403880454c64050251612997076a"}
Nov 04 01:16:47.661191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.660721    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" event={"ID":"c4bffc01-0d12-438e-aaec-7873e04b837d","Type":"ContainerStarted","Data":"6fee771903ed914b09b0c42ba05decb18b2d1af1a5d2756906342b322aa910f6"}
Nov 04 01:16:47.780201 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.780143    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podStartSLOduration=1.780126257 podStartE2EDuration="1.780126257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:47.77757823 +0000 UTC m=+1176.439718291" watchObservedRunningTime="2026-11-04 01:16:47.780126257 +0000 UTC m=+1176.442266318"
Nov 04 01:16:47.925609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.925527    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:47.999161 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:47.999126    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:48.322989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.322900    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"]
Nov 04 01:16:48.326742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.326717    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.389617 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.389576    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"]
Nov 04 01:16:48.430214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.430182    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-e65b23-openshift-default-dockercfg-h6b8n\""
Nov 04 01:16:48.440407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440382    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440432    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440593 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440549    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440640 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440606    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440755 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440738    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440810    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440888 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440847    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxjr7\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.440969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.440945    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.441027 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.441008    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.541913 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.541873    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.541929    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.541970    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.541989    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxjr7\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542021    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542039    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542091    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542133    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542198    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542401    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542592 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542567    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542630    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542732 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542658    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.542839 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.542817    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.544491 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.544469    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.544706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.544686    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.599337 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.599266    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.599507 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.599483    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxjr7\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.640134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.640106    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:48.926748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.926718    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 04 01:16:48.927113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:48.926769    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 04 01:16:49.094252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.094221    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"]
Nov 04 01:16:49.097654 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:16:49.097622    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1ce881_d58b_49e8_a18e_58b36f34d299.slice/crio-d0e166690303b8d52057322205193a7e51aa85d937e4adcf62a434506052f3b5 WatchSource:0}: Error finding container d0e166690303b8d52057322205193a7e51aa85d937e4adcf62a434506052f3b5: Status 404 returned error can't find the container with id d0e166690303b8d52057322205193a7e51aa85d937e4adcf62a434506052f3b5
Nov 04 01:16:49.100336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.100296    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:49.100416 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.100390    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:49.100459 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.100431    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:16:49.697891 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.697851    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" event={"ID":"6c1ce881-d58b-49e8-a18e-58b36f34d299","Type":"ContainerStarted","Data":"ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc"}
Nov 04 01:16:49.698102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.697908    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" event={"ID":"6c1ce881-d58b-49e8-a18e-58b36f34d299","Type":"ContainerStarted","Data":"d0e166690303b8d52057322205193a7e51aa85d937e4adcf62a434506052f3b5"}
Nov 04 01:16:49.815246 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.815191    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podStartSLOduration=1.815177472 podStartE2EDuration="1.815177472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:16:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:16:49.814653117 +0000 UTC m=+1178.476793176" watchObservedRunningTime="2026-11-04 01:16:49.815177472 +0000 UTC m=+1178.477317532"
Nov 04 01:16:49.925072 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.925033    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 04 01:16:49.925284 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:49.925100    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 04 01:16:50.641205 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:50.641171    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:50.642677 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:50.642639    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 04 01:16:50.642787 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:50.642715    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 04 01:16:50.925897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:50.925815    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 04 01:16:50.925897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:50.925885    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 04 01:16:51.640951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:51.640917    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 04 01:16:51.641128 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:51.640969    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 04 01:16:51.925473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:51.925326    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 04 01:16:51.925473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:51.925389    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 04 01:16:51.948581 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:16:51.935789    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:16:52.641335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:52.641298    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 04 01:16:52.641510 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:52.641365    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 04 01:16:52.925168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:52.925067    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 04 01:16:52.925168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:52.925137    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 04 01:16:53.014150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:53.014098    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:53.641349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:53.641310    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 04 01:16:53.641557 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:53.641369    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 04 01:16:54.160080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:54.160050    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:54.160576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:54.160427    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:54.161404 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:54.161385    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:16:54.640541 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:54.640506    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 04 01:16:54.640733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:54.640559    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 04 01:16:55.640637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:55.640602    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Nov 04 01:16:55.641045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:55.640684    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Nov 04 01:16:56.645043 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:56.645015    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:56.645429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:56.645395    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:56.645977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:56.645961    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:16:57.982295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:57.982260    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:16:59.788145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:16:59.788112    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:00.581805 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:00.581729    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:00.774377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:00.774341    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:01.229131 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:01.229098    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:02.697783 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:02.697752    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:02.897891 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:02.897855    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:07.801439 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:07.801397    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:09.521073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:09.521039    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:12.696158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:12.696123    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:15.038118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:15.038082    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:17.608213 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:17.608150    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:19.210100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.210066    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"]
Nov 04 01:17:19.214176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.214159    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.266869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.266840    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e65b23\"/\"distinct-w4-e65b23-kserve-self-signed-certs\""
Nov 04 01:17:19.269331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.269302    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"]
Nov 04 01:17:19.311594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.311565    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.311754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.311617    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.311754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.311642    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.311837 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.311812    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.311871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.311848    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.311871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.311864    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqz9t\" (UniqueName: \"kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.322081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.322054    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:19.413271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413233    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413445 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413283    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413445 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413311    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqz9t\" (UniqueName: \"kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413445 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413425    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413590 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413475    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413590 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413497    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413788    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413885 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413821    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.413885 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.413875    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.415809 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.415784    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.415959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.415941    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.476741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.471741    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqz9t\" (UniqueName: \"kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t\") pod \"distinct-w4-e65b23-kserve-746968d85d-hmp4b\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") " pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.525804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.525757    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:19.903356 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:19.903328    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"]
Nov 04 01:17:19.905980 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:17:19.905950    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfde8002a_e824_4850_9581_1f7d62bd7a59.slice/crio-e9c123b68d460eecdbc349313e0a2e53551e2d9fa5bd51c6051200d1b7b7657f WatchSource:0}: Error finding container e9c123b68d460eecdbc349313e0a2e53551e2d9fa5bd51c6051200d1b7b7657f: Status 404 returned error can't find the container with id e9c123b68d460eecdbc349313e0a2e53551e2d9fa5bd51c6051200d1b7b7657f
Nov 04 01:17:20.874158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:20.874113    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" event={"ID":"fde8002a-e824-4850-9581-1f7d62bd7a59","Type":"ContainerStarted","Data":"4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002"}
Nov 04 01:17:20.874158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:20.874162    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" event={"ID":"fde8002a-e824-4850-9581-1f7d62bd7a59","Type":"ContainerStarted","Data":"e9c123b68d460eecdbc349313e0a2e53551e2d9fa5bd51c6051200d1b7b7657f"}
Nov 04 01:17:20.990940 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:20.990889    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podStartSLOduration=1.9908752509999998 podStartE2EDuration="1.990875251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:20.988892632 +0000 UTC m=+1209.651032695" watchObservedRunningTime="2026-11-04 01:17:20.990875251 +0000 UTC m=+1209.653015312"
Nov 04 01:17:21.526652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:21.526621    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:21.897690 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:17:21.897640    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:17:22.487050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:22.487017    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:27.384823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:27.384756    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:29.525914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:29.525883    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:29.538767 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:29.538743    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:29.940812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:29.940774    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:17:32.408714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:32.408680    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:33.416801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:33.416768    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:37.287629 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:37.287595    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:39.019643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.019609    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"]
Nov 04 01:17:39.023257 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.023239    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.078684 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.078624    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"]
Nov 04 01:17:39.122619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.122584    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.122619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.122622    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.122821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.122641    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.122821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.122812    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.122887 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.122836    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvtlg\" (UniqueName: \"kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.122923 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.122882    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.125631 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.125610    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ee045\"/\"distinct-w3-2ee045-kserve-self-signed-certs\""
Nov 04 01:17:39.125727 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.125635    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ee045\"/\"default-dockercfg-pjk8b\""
Nov 04 01:17:39.126018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.126001    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2ee045\"/\"kube-root-ca.crt\""
Nov 04 01:17:39.126071 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.126059    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-2ee045\"/\"openshift-service-ca.crt\""
Nov 04 01:17:39.137081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.137062    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:39.224570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.224539    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.224779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.224646    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.224779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.224695    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.224779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.224713    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.224779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.224774    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.225049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.224798    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvtlg\" (UniqueName: \"kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.225143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.225115    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.225412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.225388    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.225412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.225402    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.227256 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.227212    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.228607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.228562    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.281907 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.281835    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvtlg\" (UniqueName: \"kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg\") pod \"distinct-w3-2ee045-kserve-77db54895c-gj87z\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.336501 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.336459    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:39.721807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.721770    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"]
Nov 04 01:17:39.725244 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:17:39.725209    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fff6230_5d60_49e0_b116_3f060d15a052.slice/crio-501253abbccf67aef1c1a350640c8eae760d797e9172ee3994bd1c079f184af0 WatchSource:0}: Error finding container 501253abbccf67aef1c1a350640c8eae760d797e9172ee3994bd1c079f184af0: Status 404 returned error can't find the container with id 501253abbccf67aef1c1a350640c8eae760d797e9172ee3994bd1c079f184af0
Nov 04 01:17:39.978710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.978616    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" event={"ID":"9fff6230-5d60-49e0-b116-3f060d15a052","Type":"ContainerStarted","Data":"f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7"}
Nov 04 01:17:39.978710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:39.978659    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" event={"ID":"9fff6230-5d60-49e0-b116-3f060d15a052","Type":"ContainerStarted","Data":"501253abbccf67aef1c1a350640c8eae760d797e9172ee3994bd1c079f184af0"}
Nov 04 01:17:40.094685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:40.093830    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podStartSLOduration=2.093811012 podStartE2EDuration="2.093811012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:17:40.09217119 +0000 UTC m=+1228.754311254" watchObservedRunningTime="2026-11-04 01:17:40.093811012 +0000 UTC m=+1228.755951074"
Nov 04 01:17:40.336785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:40.336660    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:42.177608 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:42.177566    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:47.092916 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:47.092861    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:49.336974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:49.336940    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:49.349168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:49.349143    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:50.040067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:50.040031    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:17:51.897351 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:17:51.897314    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:17:51.986379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:51.986350    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:17:57.058715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:17:57.058684    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:01.626477 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.626432    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:01.677572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.677540    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"]
Nov 04 01:18:01.681410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.681386    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.736897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.736855    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-2fd27d\"/\"distinct-2-w2-2fd27d-kserve-self-signed-certs\""
Nov 04 01:18:01.740116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.740090    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"]
Nov 04 01:18:01.782711 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.782647    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.782886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.782716    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.782886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.782787    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks2mn\" (UniqueName: \"kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.782886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.782841    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.783025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.782999    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.783060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.783038    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.872699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.872647    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:01.884314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884227    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884269    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884307    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884327    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884361    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks2mn\" (UniqueName: \"kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884425    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884845    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.884910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884871    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.885102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.884932    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.886762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.886740    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.886953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.886936    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.948001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.947963    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks2mn\" (UniqueName: \"kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn\") pod \"distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") " pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:01.994722 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:01.994694    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:02.105412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:02.105332    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:02.383566 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:02.383534    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"]
Nov 04 01:18:02.387649 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:18:02.387618    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod353b1b1b_6900_4caf_88c4_932744856777.slice/crio-54b93ad153e0d636f5a800b1c80f0b5e66970a4f1c53443f8822a85dc5b8e229 WatchSource:0}: Error finding container 54b93ad153e0d636f5a800b1c80f0b5e66970a4f1c53443f8822a85dc5b8e229: Status 404 returned error can't find the container with id 54b93ad153e0d636f5a800b1c80f0b5e66970a4f1c53443f8822a85dc5b8e229
Nov 04 01:18:03.118804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:03.118764    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" event={"ID":"353b1b1b-6900-4caf-88c4-932744856777","Type":"ContainerStarted","Data":"5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c"}
Nov 04 01:18:03.118804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:03.118809    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" event={"ID":"353b1b1b-6900-4caf-88c4-932744856777","Type":"ContainerStarted","Data":"54b93ad153e0d636f5a800b1c80f0b5e66970a4f1c53443f8822a85dc5b8e229"}
Nov 04 01:18:03.242290 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:03.242233    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podStartSLOduration=2.242218301 podStartE2EDuration="2.242218301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:03.238465046 +0000 UTC m=+1251.900605106" watchObservedRunningTime="2026-11-04 01:18:03.242218301 +0000 UTC m=+1251.904358362"
Nov 04 01:18:03.995536 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:03.995505    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:07.019474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:07.019430    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:11.995618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:11.995587    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:12.007847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:12.007823    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:12.085364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:12.085339    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:12.173860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:12.173828    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:18:17.125843 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:17.125799    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:18.276096 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.276057    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"]
Nov 04 01:18:18.280255 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.280236    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.333364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.333336    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-c563ee-openshift-default-dockercfg-5mps8\""
Nov 04 01:18:18.384120 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384082    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384186    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384227    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384255    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384304    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384331    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384546 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384473    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384546 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384521    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.384708 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.384548    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swhvm\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.387016 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.386993    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"]
Nov 04 01:18:18.485900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.485868    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swhvm\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.485911    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.485959    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.485983    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486002    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486047    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486075    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486211    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486259    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486410    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486422    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486677 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486639    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486681    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.486887 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.486862    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.488601 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.488581    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.488731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.488716    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.543780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.543716    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.544940 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.544919    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swhvm\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:18.593574 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:18.593545    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:19.027831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.027798    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"]
Nov 04 01:18:19.071383 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:18:19.071357    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ace3819_81cc_4578_9d47_108614bb3cf7.slice/crio-04e46864495d006899c1943daeb319ac02a0061a0b7bf2e9aedfea3df01d65b2 WatchSource:0}: Error finding container 04e46864495d006899c1943daeb319ac02a0061a0b7bf2e9aedfea3df01d65b2: Status 404 returned error can't find the container with id 04e46864495d006899c1943daeb319ac02a0061a0b7bf2e9aedfea3df01d65b2
Nov 04 01:18:19.073845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.073812    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:18:19.073908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.073884    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:18:19.073951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.073915    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:18:19.201174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.201141    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" event={"ID":"8ace3819-81cc-4578-9d47-108614bb3cf7","Type":"ContainerStarted","Data":"59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5"}
Nov 04 01:18:19.201345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.201197    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" event={"ID":"8ace3819-81cc-4578-9d47-108614bb3cf7","Type":"ContainerStarted","Data":"04e46864495d006899c1943daeb319ac02a0061a0b7bf2e9aedfea3df01d65b2"}
Nov 04 01:18:19.363006 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.362900    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podStartSLOduration=1.3628859979999999 podStartE2EDuration="1.362885998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:18:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:18:19.359424977 +0000 UTC m=+1268.021565037" watchObservedRunningTime="2026-11-04 01:18:19.362885998 +0000 UTC m=+1268.025026059"
Nov 04 01:18:19.593826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:19.593794    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:20.595576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:20.595545    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 01:18:20.596071 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:20.595596    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 01:18:21.560000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:21.559964    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:21.594900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:21.594862    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 01:18:21.595074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:21.594924    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 01:18:21.917201 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:18:21.917163    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:18:22.128568 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:22.128537    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:22.594765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:22.594729    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 01:18:22.594951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:22.594801    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 01:18:23.594342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:23.594307    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 01:18:23.594896 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:23.594375    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 01:18:24.594292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:24.594259    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 01:18:24.594468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:24.594311    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 01:18:25.594455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:25.594418    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 04 01:18:25.594847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:25.594490    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 04 01:18:26.599164 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:26.599133    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:26.599635 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:26.599621    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:26.600210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:26.600189    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:18:27.270561 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:27.270527    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:32.104469 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:32.104432    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:37.111890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:37.111849    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:41.994280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:41.994248    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:45.807318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:45.807285    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:46.918898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:46.918866    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:51.899517 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:18:51.899485    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:18:51.904267 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:51.904242    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:18:57.105501 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:18:57.105319    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:01.923265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:01.923217    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:02.092861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:02.092829    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:05.011538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:05.011506    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:07.136955 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:07.136920    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:12.101335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:12.101299    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:17.388600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:17.388561    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:21.899026 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:19:21.898984    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:19:22.370143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:22.370055    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:27.357811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:27.357776    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:28.679990 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:28.679939    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:32.315276 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:32.315238    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:38.682576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:38.682544    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:42.299907 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:42.299861    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:43.122104 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:43.122067    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:43.296208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:43.296177    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:44.724775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:44.724744    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:47.291575 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:47.291540    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:48.387994 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.387959    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"]
Nov 04 01:19:48.392061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.392012    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.500530 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.500495    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"]
Nov 04 01:19:48.522395 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522369    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522536 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522407    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522536 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522500    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522544    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522655    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522711    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8862h\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522741    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.522890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522844    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.523007 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.522910    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.623637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623606    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.623637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623638    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8862h\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.623910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623661    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.623910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623724    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.623910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623819    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.623910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623881    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623916    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.623986    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.624073    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.624116    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.624145    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.624235    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.624322    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.624656 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.624633    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.626062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.626040    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.626398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.626381    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.682827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.682762    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8862h\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.683086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.683067    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:48.705018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:48.704995    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:49.133409 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.133374    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"]
Nov 04 01:19:49.176304 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:19:49.176274    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8ed7bb_946b_496f_ae42_a632f7b7d8cd.slice/crio-d51deb91a0b7ccc238fc8eb63463ad070e42bb8c5544411bc8c96d131754108b WatchSource:0}: Error finding container d51deb91a0b7ccc238fc8eb63463ad070e42bb8c5544411bc8c96d131754108b: Status 404 returned error can't find the container with id d51deb91a0b7ccc238fc8eb63463ad070e42bb8c5544411bc8c96d131754108b
Nov 04 01:19:49.178584 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.178555    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:19:49.178643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.178618    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:19:49.178702 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.178644    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:19:49.697210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.697161    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" event={"ID":"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd","Type":"ContainerStarted","Data":"d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9"}
Nov 04 01:19:49.697700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.697243    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" event={"ID":"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd","Type":"ContainerStarted","Data":"d51deb91a0b7ccc238fc8eb63463ad070e42bb8c5544411bc8c96d131754108b"}
Nov 04 01:19:49.705435 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.705411    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:49.919890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:49.919831    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podStartSLOduration=1.9198183370000002 podStartE2EDuration="1.919818337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:19:49.868120839 +0000 UTC m=+1358.530260902" watchObservedRunningTime="2026-11-04 01:19:49.919818337 +0000 UTC m=+1358.581958472"
Nov 04 01:19:50.706748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:50.706711    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 01:19:50.707139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:50.706787    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 01:19:51.705826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:51.705791    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 01:19:51.705999 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:51.705847    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 01:19:51.917203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:51.917166    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:51.922194 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:19:51.922161    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:19:52.416346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:52.416310    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:52.705652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:52.705572    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 01:19:52.705652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:52.705634    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 01:19:53.705618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:53.705585    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 01:19:53.706044 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:53.705648    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 01:19:54.705991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:54.705959    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 01:19:54.706380 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:54.706020    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 01:19:55.705589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:55.705554    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 04 01:19:55.705793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:55.705615    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 04 01:19:56.709850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:56.709816    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:56.710243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:56.710211    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:56.710792 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:56.710775    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:19:57.437729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:57.437694    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:19:57.623160 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:19:57.623112    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:02.278535 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:02.278497    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:02.435109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:02.435059    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:03.520869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.520830    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"]
Nov 04 01:20:03.524847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.524828    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.582972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.582938    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"]
Nov 04 01:20:03.633364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633324    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633375    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633392    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633531    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633683 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633566    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633732 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633698    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633768 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633753    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633809 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633788    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsv8m\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.633863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.633846    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.735508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735468    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.735717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735527    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.735717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735577    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.735717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735616    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.735717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735654    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsv8m\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.735717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735692    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735813    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.735902    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.736091    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.736109    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.736125    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736322 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.736179    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736409 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.736387    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.736652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.736634    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.738466 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.738439    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.738576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.738558    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.793725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.793617    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.793864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.793731    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsv8m\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m\") pod \"e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") " pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:03.838490 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:03.838451    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:04.273721 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.273691    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"]
Nov 04 01:20:04.276648 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:20:04.276621    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9024f494_9faf_4e2b_9fe1_eb33b94ca551.slice/crio-b63596965fb415b591fcda8ffe8163ca9ea11e1bf37d7bba9c12ad4bf4625ea4 WatchSource:0}: Error finding container b63596965fb415b591fcda8ffe8163ca9ea11e1bf37d7bba9c12ad4bf4625ea4: Status 404 returned error can't find the container with id b63596965fb415b591fcda8ffe8163ca9ea11e1bf37d7bba9c12ad4bf4625ea4
Nov 04 01:20:04.279381 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.279343    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:20:04.279480 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.279449    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:20:04.279543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.279497    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:20:04.783605 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.783562    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" event={"ID":"9024f494-9faf-4e2b-9fe1-eb33b94ca551","Type":"ContainerStarted","Data":"d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d"}
Nov 04 01:20:04.784134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.783627    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" event={"ID":"9024f494-9faf-4e2b-9fe1-eb33b94ca551","Type":"ContainerStarted","Data":"b63596965fb415b591fcda8ffe8163ca9ea11e1bf37d7bba9c12ad4bf4625ea4"}
Nov 04 01:20:04.839378 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.839340    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:04.952581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:04.952531    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podStartSLOduration=1.9525155189999999 podStartE2EDuration="1.952515519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:04.950567131 +0000 UTC m=+1373.612707191" watchObservedRunningTime="2026-11-04 01:20:04.952515519 +0000 UTC m=+1373.614655579"
Nov 04 01:20:05.513128 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.513087    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"]
Nov 04 01:20:05.517178 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.517160    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.643922 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.643883    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:05.645364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.645344    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-2ee045\"/\"distinct-2-w3-2ee045-kserve-self-signed-certs\""
Nov 04 01:20:05.648181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.648155    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"]
Nov 04 01:20:05.662964 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.662929    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.663313 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.663290    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.663534 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.663517    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fff6c\" (UniqueName: \"kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.663738 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.663721    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.669704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.663924    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.675904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.675870    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.780500    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.780600    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.780695    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.780760    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.780862    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fff6c\" (UniqueName: \"kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.780918    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.781397    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.781654    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.783113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.782255    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.799693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.797458    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.801687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.801641    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.840601 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.840567    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fff6c\" (UniqueName: \"kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c\") pod \"distinct-2-w3-2ee045-kserve-66866f894-l4hh9\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") " pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:05.840803 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.840702    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 01:20:05.840803 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:05.840747    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 01:20:06.008657 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.008618    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:06.130480 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.130437    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:06.544149 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.543949    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"]
Nov 04 01:20:06.546979 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:20:06.546942    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a3ab8da_14e4_4a0f_8300_88850af83969.slice/crio-a3fb4b06bba6ea4c52ea1c5f12056e2ad88bbd8b6a9a8fa3b366e3f47ab8f039 WatchSource:0}: Error finding container a3fb4b06bba6ea4c52ea1c5f12056e2ad88bbd8b6a9a8fa3b366e3f47ab8f039: Status 404 returned error can't find the container with id a3fb4b06bba6ea4c52ea1c5f12056e2ad88bbd8b6a9a8fa3b366e3f47ab8f039
Nov 04 01:20:06.808965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.808863    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" event={"ID":"4a3ab8da-14e4-4a0f-8300-88850af83969","Type":"ContainerStarted","Data":"fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a"}
Nov 04 01:20:06.808965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.808910    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" event={"ID":"4a3ab8da-14e4-4a0f-8300-88850af83969","Type":"ContainerStarted","Data":"a3fb4b06bba6ea4c52ea1c5f12056e2ad88bbd8b6a9a8fa3b366e3f47ab8f039"}
Nov 04 01:20:06.839880 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.839848    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 01:20:06.840048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.839907    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 01:20:06.937639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:06.937582    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podStartSLOduration=1.937566281 podStartE2EDuration="1.937566281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:06.925090904 +0000 UTC m=+1375.587230967" watchObservedRunningTime="2026-11-04 01:20:06.937566281 +0000 UTC m=+1375.599706342"
Nov 04 01:20:07.131232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:07.131189    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:07.542346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:07.542264    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:07.839868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:07.839779    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 01:20:07.839868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:07.839833    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 01:20:08.839827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:08.839793    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 01:20:08.840024 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:08.839851    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 01:20:09.839704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:09.839594    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 01:20:09.839704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:09.839652    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 01:20:10.839461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:10.839419    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 04 01:20:10.839942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:10.839777    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 04 01:20:12.551868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:12.551830    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:12.552380 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:12.552261    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:12.553256 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:12.553233    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:20:14.215379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:14.215300    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:16.130798 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:16.130760    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:16.144992 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:16.144961    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:16.910951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:16.910917    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:20:19.313950 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:19.313897    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:20.053769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:20.053734    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"]
Nov 04 01:20:20.103462 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:20.103426    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0801cec9-sv57f"]
Nov 04 01:20:21.475573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:21.475534    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f434e13b-c5a4-4d82-a067-278aaa95cc4b" path="/var/lib/kubelet/pods/f434e13b-c5a4-4d82-a067-278aaa95cc4b/volumes"
Nov 04 01:20:21.898344 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:20:21.898307    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:20:24.282976 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:24.282896    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:29.260203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:29.260134    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:34.198446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:34.198412    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:39.191733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:39.191701    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:44.188393 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:44.188363    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:49.594189 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:49.594144    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:50.164359 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.164325    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"]
Nov 04 01:20:50.168487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.168470    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.276684 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.276626    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"]
Nov 04 01:20:50.279270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279235    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279297    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279499 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279423    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279499 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279462    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279612 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279527    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279612 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279559    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5psvz\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279611    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279683    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.279758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.279727    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381020 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.380983    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381028    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381059    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381082    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381136    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381166    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5psvz\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381225    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381456    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381467    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381526    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381586    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381614    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.381804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.381744    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.382325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.382295    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.383795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.383764    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.383901 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.383853    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.439317 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.439249    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.439317 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.439308    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5psvz\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.481754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.481724    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:50.914281 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.914254    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"]
Nov 04 01:20:50.916524 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:20:50.916497    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49dd45c_5812_43e4_a378_efb30ce97358.slice/crio-ae1dc246f02de6ea3b52d9e847508bc9db3bad20e56bf02beb87d492276b158a WatchSource:0}: Error finding container ae1dc246f02de6ea3b52d9e847508bc9db3bad20e56bf02beb87d492276b158a: Status 404 returned error can't find the container with id ae1dc246f02de6ea3b52d9e847508bc9db3bad20e56bf02beb87d492276b158a
Nov 04 01:20:50.918808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.918779    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:20:50.918897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.918840    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:20:50.918897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:50.918872    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:20:51.120412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:51.120380    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" event={"ID":"d49dd45c-5812-43e4-a378-efb30ce97358","Type":"ContainerStarted","Data":"d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae"}
Nov 04 01:20:51.120608 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:51.120438    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" event={"ID":"d49dd45c-5812-43e4-a378-efb30ce97358","Type":"ContainerStarted","Data":"ae1dc246f02de6ea3b52d9e847508bc9db3bad20e56bf02beb87d492276b158a"}
Nov 04 01:20:51.242727 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:51.242614    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podStartSLOduration=1.24259916 podStartE2EDuration="1.24259916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:20:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:20:51.239496424 +0000 UTC m=+1419.901636485" watchObservedRunningTime="2026-11-04 01:20:51.24259916 +0000 UTC m=+1419.904739220"
Nov 04 01:20:51.481994 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:51.481960    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:51.483578 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:51.483552    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:51.483746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:51.483608    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:51.917547 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:20:51.917511    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:20:52.488243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:52.488202    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:52.488567 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:52.488533    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:53.489115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:53.489077    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:53.489627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:53.489154    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:54.485691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:54.483983    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:54.485691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:54.484043    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:54.605561 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:54.605529    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:20:55.489696 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:55.485692    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:55.489696 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:55.485759    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:56.484096 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:56.484054    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:56.484735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:56.484704    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:57.483000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:57.482967    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 04 01:20:57.483214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:57.483016    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 04 01:20:58.487037 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:58.487002    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:58.487458 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:58.487345    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:58.488204 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:58.488181    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:20:59.594791 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:20:59.594756    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:02.656411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:02.656369    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:04.536319 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:04.536274    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:04.936199 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:04.936158    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"]
Nov 04 01:21:04.939828 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:04.939809    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:04.998549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:04.998519    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"]
Nov 04 01:21:05.045014 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.044980    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-2ee045-openshift-default-dockercfg-2pv54\""
Nov 04 01:21:05.081516 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081485    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.081705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081532    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.081705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081611    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.081705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081642    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.081851 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081761    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.081851 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081784    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.081851 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081820    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.082010 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.081993    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.082113 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.082095    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqqwn\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.183675 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.183637    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.183892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.183748    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqqwn\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.183892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.183820    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.183892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.183854    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.183892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.183888    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.183912    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184107    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184158    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184222    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184343    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184440 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184409    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184480    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184479    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.184830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.184808    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.186347 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.186294    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.186451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.186394    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.242033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.241998    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.242191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.242156    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqqwn\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.253881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.253860    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:05.687877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.687848    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"]
Nov 04 01:21:05.689042 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:21:05.689018    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode58fd0fd_8758_4cae_ba36_96ecc6989b60.slice/crio-d85a6b758bdcdd13921d33ed6b7218e249886472b9d58af207bffb7356b4f659 WatchSource:0}: Error finding container d85a6b758bdcdd13921d33ed6b7218e249886472b9d58af207bffb7356b4f659: Status 404 returned error can't find the container with id d85a6b758bdcdd13921d33ed6b7218e249886472b9d58af207bffb7356b4f659
Nov 04 01:21:05.691497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.691456    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:05.691584 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.691538    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:05.691630 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:05.691582    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:06.213432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:06.213396    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" event={"ID":"e58fd0fd-8758-4cae-ba36-96ecc6989b60","Type":"ContainerStarted","Data":"5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c"}
Nov 04 01:21:06.213611 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:06.213446    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" event={"ID":"e58fd0fd-8758-4cae-ba36-96ecc6989b60","Type":"ContainerStarted","Data":"d85a6b758bdcdd13921d33ed6b7218e249886472b9d58af207bffb7356b4f659"}
Nov 04 01:21:06.254646 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:06.254620    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:06.256436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:06.256414    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:06.256495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:06.256459    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:06.336479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:06.336423    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podStartSLOduration=2.336408156 podStartE2EDuration="2.336408156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:06.334492711 +0000 UTC m=+1434.996632773" watchObservedRunningTime="2026-11-04 01:21:06.336408156 +0000 UTC m=+1434.998548216"
Nov 04 01:21:07.255213 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:07.255177    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:07.255716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:07.255235    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:08.255088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:08.255056    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:08.255244 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:08.255108    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:09.254859 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:09.254827    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:09.255063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:09.254881    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:09.821483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:09.821429    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:10.254741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:10.254707    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:10.254922 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:10.254761    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:11.254679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:11.254629    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:11.255080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:11.254706    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:12.254895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:12.254856    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Nov 04 01:21:12.255286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:12.254923    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Nov 04 01:21:13.258403 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:13.258375    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:13.258887 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:13.258839    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:13.259596 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:13.259575    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:21:14.887514 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:14.887481    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:19.359772 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.359736    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"]
Nov 04 01:21:19.364881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.364858    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.446313 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.446260    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"]
Nov 04 01:21:19.496636 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.496602    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.496938 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.496915    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.497104 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.497085    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.497264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.497249    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.497411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.497397    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.497512 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.497499    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.497800 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.497779    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.498013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.497997    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.498171 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.498157    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn2pv\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.599859 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.599818    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.599902    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.599943    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.599993    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600081    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600133    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn2pv\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600270 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600199    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600293    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600383    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.600884 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600816    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.601016 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.600932    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.601147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.601119    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.601383 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.601359    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.602156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.601468    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.603489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.603467    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.604809 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.604785    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.671483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.671443    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.674135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.674099    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn2pv\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:19.678938 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:19.678906    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:20.241077 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:20.233092    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"]
Nov 04 01:21:20.254812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:20.253477    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:20.254812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:20.253570    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:20.254812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:20.253614    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:20.327340 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:20.321605    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:20.329192 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:20.329101    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" event={"ID":"3e582e99-2e83-43a3-b470-5929d857ffe8","Type":"ContainerStarted","Data":"cca491e75dc9a8506090181cd416258ecea899f39c543bb2f3829a39dd426d34"}
Nov 04 01:21:21.414128 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:21.414082    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" event={"ID":"3e582e99-2e83-43a3-b470-5929d857ffe8","Type":"ContainerStarted","Data":"8a94fb4ea22c838477362ac4cc24cdd78dc950c815e85ccc5fd08c306b37e10b"}
Nov 04 01:21:21.673822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:21.672022    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podStartSLOduration=2.672001687 podStartE2EDuration="2.672001687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:21.616128827 +0000 UTC m=+1450.278268889" watchObservedRunningTime="2026-11-04 01:21:21.672001687 +0000 UTC m=+1450.334141749"
Nov 04 01:21:21.690929 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:21.690839    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:21.729699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:21.729645    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:21.729967 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:21.729928    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:22.192946 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:21:22.192902    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:21:22.687688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:22.687152    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:22.687688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:22.687255    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:23.689208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:23.689168    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:23.689726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:23.689254    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:24.680163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:24.680115    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:24.688832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:24.688772    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:25.684690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:25.684129    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:25.684690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:25.684206    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:26.194690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.192952    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"]
Nov 04 01:21:26.194690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.193212    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy" containerID="cri-o://838b109e6bfb62ad262710e0c06ea33ca302b994584b3ee04cc2b9124eeeca02" gracePeriod=30
Nov 04 01:21:26.237188 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.237153    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"]
Nov 04 01:21:26.237368 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.237349    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy" containerID="cri-o://5e2be7fe2673c4a320df537a6f446b4f4c1b9ff75bb0e9a7e0c9244a74721dcb" gracePeriod=30
Nov 04 01:21:26.530569 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.530491    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:26.684689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.682047    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:26.684689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:26.682111    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:27.679902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:27.679858    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Nov 04 01:21:27.680100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:27.679943    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Nov 04 01:21:28.803846 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:28.803751    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:28.804497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:28.804439    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:28.805206 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:28.805188    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:21:31.486169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.486125    4434 generic.go:356] "Generic (PLEG): container finished" podID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerID="838b109e6bfb62ad262710e0c06ea33ca302b994584b3ee04cc2b9124eeeca02" exitCode=0
Nov 04 01:21:31.486681 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.486265    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" event={"ID":"1889a421-b757-4bad-8b15-f8ff87ae8e9e","Type":"ContainerDied","Data":"838b109e6bfb62ad262710e0c06ea33ca302b994584b3ee04cc2b9124eeeca02"}
Nov 04 01:21:31.488227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.488200    4434 generic.go:356] "Generic (PLEG): container finished" podID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerID="5e2be7fe2673c4a320df537a6f446b4f4c1b9ff75bb0e9a7e0c9244a74721dcb" exitCode=0
Nov 04 01:21:31.488356 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.488311    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" event={"ID":"3bd4e2be-7854-48c2-9cf3-09425a736b43","Type":"ContainerDied","Data":"5e2be7fe2673c4a320df537a6f446b4f4c1b9ff75bb0e9a7e0c9244a74721dcb"}
Nov 04 01:21:31.515547 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.515504    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:21:31.551497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.551467    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:21:31.615779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.615745    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.615953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.615786    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.615953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.615811    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616016    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616075    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.616168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616161    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616218    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616317 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616278    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.616317 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616286    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616323    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616352    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616393    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616429    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616467    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616516    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616548    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616588    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616615    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616467    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.616766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616575    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.616927 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616786    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket\") pod \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\" (UID: \"1889a421-b757-4bad-8b15-f8ff87ae8e9e\") "
Nov 04 01:21:31.616927 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616852    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert\") pod \"3bd4e2be-7854-48c2-9cf3-09425a736b43\" (UID: \"3bd4e2be-7854-48c2-9cf3-09425a736b43\") "
Nov 04 01:21:31.616927 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.616884    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.617058 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617003    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:21:31.617470 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617444    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.617572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617474    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.617572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617475    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.617572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617488    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.617572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617528    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.617572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617546    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.617572 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617561    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.617942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617846    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.617942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.617868    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:21:31.618071 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.618049    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.618563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.618528    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "kube-api-access-gwgmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:31.619126 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.619090    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:31.619516 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.619489    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.619617 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.619521    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "kube-api-access-gqjt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:31.620920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.620858    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:21:31.620920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.620899    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:21:31.621083 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.620945    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo" pod "1889a421-b757-4bad-8b15-f8ff87ae8e9e" (UID: "1889a421-b757-4bad-8b15-f8ff87ae8e9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:21:31.621083 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.620952    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo" pod "3bd4e2be-7854-48c2-9cf3-09425a736b43" (UID: "3bd4e2be-7854-48c2-9cf3-09425a736b43"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:21:31.632241 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.632213    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719220    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719246    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3bd4e2be-7854-48c2-9cf3-09425a736b43-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719256    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwgmj\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-kube-api-access-gwgmj\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719266    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719275    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719283    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719292    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqjt5\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-kube-api-access-gqjt5\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719301    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719309    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1889a421-b757-4bad-8b15-f8ff87ae8e9e-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719317    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719325    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3bd4e2be-7854-48c2-9cf3-09425a736b43-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:31.719599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:31.719334    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3bd4e2be-7854-48c2-9cf3-09425a736b43-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:21:32.499199 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.499124    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"
Nov 04 01:21:32.499199 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.499139    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v" event={"ID":"1889a421-b757-4bad-8b15-f8ff87ae8e9e","Type":"ContainerDied","Data":"1fe561679c5f697c418d33faac4fc39e34100b1a73502da6043141632f339d8b"}
Nov 04 01:21:32.499729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.499207    4434 scope.go:122] "RemoveContainer" containerID="838b109e6bfb62ad262710e0c06ea33ca302b994584b3ee04cc2b9124eeeca02"
Nov 04 01:21:32.501727 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.501701    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls" event={"ID":"3bd4e2be-7854-48c2-9cf3-09425a736b43","Type":"ContainerDied","Data":"0c4d98f16824d6b68a7312cecc7223fa5d0369b759f1351ea91caa8a01e10a2c"}
Nov 04 01:21:32.501727 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.501725    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"
Nov 04 01:21:32.514181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.514161    4434 scope.go:122] "RemoveContainer" containerID="5e2be7fe2673c4a320df537a6f446b4f4c1b9ff75bb0e9a7e0c9244a74721dcb"
Nov 04 01:21:32.716188 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.716145    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"]
Nov 04 01:21:32.766025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.765953    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-lxc9v"]
Nov 04 01:21:32.828614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.828580    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"]
Nov 04 01:21:32.832980 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:32.832957    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-m9nls"]
Nov 04 01:21:33.474971 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:33.474936    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" path="/var/lib/kubelet/pods/1889a421-b757-4bad-8b15-f8ff87ae8e9e/volumes"
Nov 04 01:21:33.475521 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:33.475503    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" path="/var/lib/kubelet/pods/3bd4e2be-7854-48c2-9cf3-09425a736b43/volumes"
Nov 04 01:21:36.610104 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:36.610061    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:41.949827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:41.949792    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:46.899218 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:46.899178    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:48.569785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.569752    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"]
Nov 04 01:21:48.570377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.570350    4434 cpu_manager.go:414] "containerMap: removing container" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy"
Nov 04 01:21:48.570377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.570369    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy"
Nov 04 01:21:48.570454 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.570385    4434 cpu_manager.go:414] "containerMap: removing container" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy"
Nov 04 01:21:48.570454 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.570391    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy"
Nov 04 01:21:48.570515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.570489    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bd4e2be-7854-48c2-9cf3-09425a736b43" containerName="istio-proxy"
Nov 04 01:21:48.570515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.570500    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="1889a421-b757-4bad-8b15-f8ff87ae8e9e" containerName="istio-proxy"
Nov 04 01:21:48.575653 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.575636    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.633531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.633497    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"]
Nov 04 01:21:48.677914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.677880    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-e210ef-openshift-default-dockercfg-qv9pj\""
Nov 04 01:21:48.720771 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.720740    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.720894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.720785    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.720894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.720805    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.720982 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.720920    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.720982 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.720942    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.721048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.721035    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grwr5\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.721084 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.721074    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.721151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.721139    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.721209 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.721198    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822534 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822438    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822534 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822525    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822617    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822678    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822705    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822749    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.822811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822778    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822842    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grwr5\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.822906    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823167 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.823079    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823329 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.823307    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823409 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.823312    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823409 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.823395    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.823521 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.823407    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.824885 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.824863    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.825431 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.825413    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.882175 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.882150    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.882822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.882804    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grwr5\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.888398 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.888376    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:48.896649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.896617    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"]
Nov 04 01:21:48.901285 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.901265    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:48.923703 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.923657    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"]
Nov 04 01:21:48.929131 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.929104    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:48.959800 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:48.959547    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"]
Nov 04 01:21:49.005064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.005016    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"]
Nov 04 01:21:49.025235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025201    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025259    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025319    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.025414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025349    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.025414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025365    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025465    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.025594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025505    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.025594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025587    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6rfq\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025638    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025715    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025787    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025921 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025906    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.025962 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025927    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.025962 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.025948    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlzrg\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.026052 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.026004    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.026166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.026146    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.026220 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.026210    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.026311 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.026295    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127214    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127250    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.127441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127273    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlzrg\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127312    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127345    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127373    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.127441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127420    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127461    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127506    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127537    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127574    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127599    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127651    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127694    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.127784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127739    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6rfq\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127790    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127809    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.128174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127831    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127904    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.127927    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128047    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.128504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128283    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.128504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128327    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128476    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128684 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128547    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.128782 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128749    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.128842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.128758    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.129152 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.129126    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.129917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.129893    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.129917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.129911    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.130301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.130283    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.130467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.130450    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.185700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.185635    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.186192 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.186172    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.186323 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.186301    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlzrg\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.186411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.186395    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6rfq\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq\") pod \"e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") " pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.243749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.243718    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:49.248936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.248919    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:49.350712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.348961    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"]
Nov 04 01:21:49.356854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.356820    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.356961 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.356898    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.356961 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.356942    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.606370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.606329    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" event={"ID":"55f8f6a1-4e0e-4dc5-8c39-5dd939897433","Type":"ContainerStarted","Data":"7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf"}
Nov 04 01:21:49.606770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.606394    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" event={"ID":"55f8f6a1-4e0e-4dc5-8c39-5dd939897433","Type":"ContainerStarted","Data":"2d7020dd6ddb70cac9cfd4218724f9f877c1031b80c5e766d1d98e4da0511bd5"}
Nov 04 01:21:49.632486 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.632455    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"]
Nov 04 01:21:49.633336 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:21:49.633311    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9640fa9a_743b_4a7c_bc0b_88f2a1299c03.slice/crio-2d5a18ddf4fad65272943d676d0cb16ad785c1eacc7a8070f4081af6d998abdd WatchSource:0}: Error finding container 2d5a18ddf4fad65272943d676d0cb16ad785c1eacc7a8070f4081af6d998abdd: Status 404 returned error can't find the container with id 2d5a18ddf4fad65272943d676d0cb16ad785c1eacc7a8070f4081af6d998abdd
Nov 04 01:21:49.635636 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.635608    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.635738 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.635688    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.635738 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.635721    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.684460 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.684432    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"]
Nov 04 01:21:49.688238 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:21:49.688206    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f68d84c_81d2_451e_a0c5_bddcfcc625d5.slice/crio-63222f5acd8c94fe8b48b8cca6d78ee9a13179101d01baeeff2bcb311206aa15 WatchSource:0}: Error finding container 63222f5acd8c94fe8b48b8cca6d78ee9a13179101d01baeeff2bcb311206aa15: Status 404 returned error can't find the container with id 63222f5acd8c94fe8b48b8cca6d78ee9a13179101d01baeeff2bcb311206aa15
Nov 04 01:21:49.691185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.691146    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.691264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.691233    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.691303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.691278    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:49.728013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.727966    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podStartSLOduration=1.72795151 podStartE2EDuration="1.72795151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:49.723345378 +0000 UTC m=+1478.385485471" watchObservedRunningTime="2026-11-04 01:21:49.72795151 +0000 UTC m=+1478.390091571"
Nov 04 01:21:49.889371 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.889334    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:21:49.911732 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.908379    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"]
Nov 04 01:21:49.913169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.913148    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:49.973747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:49.971504    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"]
Nov 04 01:21:50.040765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.040726    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.040965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.040776    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.040965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.040882    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.041076 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.040958    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.041155 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.041124    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.041319 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.041301    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.041492 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.041463    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.041581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.041505    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slzsr\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.041581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.041543    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.142991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.142952    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143002    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slzsr\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143051    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143156    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143269    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143382 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143321    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143382 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143351    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143492 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143414    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143492 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143451    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143546    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143633    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143738 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143435    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143848 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143738    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.143959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.143927    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.145623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.145603    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.146004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.145987    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.201604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.201533    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slzsr\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.201808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.201771    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.233633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.233567    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:50.301827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.299503    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"]
Nov 04 01:21:50.307657 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.307634    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.362087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.361718    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"]
Nov 04 01:21:50.447598 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447559    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.447598 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447598    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc6dv\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.447840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447708    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.447840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447768    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.447840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447827    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.447955 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447886    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.448002 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.447990    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.448039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.448015    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.448107 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.448094    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.549650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549565    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.549650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549611    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.549867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549696    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.549867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549769    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.549867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549800    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc6dv\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.549867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549848    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550036 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549885    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550036 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.549964    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550036 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.550007    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550722 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.550696    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550888 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.550861    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.550803    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.550965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.550765    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.551342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.551320    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.552424 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.552405    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.552903 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.552886    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.622756 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.622649    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.626478 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.626443    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc6dv\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.628001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.627972    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:50.630869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.630843    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"]
Nov 04 01:21:50.634552 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:21:50.634526    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb3a9c47_d27c_4d4c_ab61_5c46aaac8aa1.slice/crio-d118fe19ddb0ef5fb0d9f6c473ecc350a78c76601a971980d2e1cb6a4674525b WatchSource:0}: Error finding container d118fe19ddb0ef5fb0d9f6c473ecc350a78c76601a971980d2e1cb6a4674525b: Status 404 returned error can't find the container with id d118fe19ddb0ef5fb0d9f6c473ecc350a78c76601a971980d2e1cb6a4674525b
Nov 04 01:21:50.644121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.644069    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:50.644495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.644386    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:50.645013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.644720    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:50.661688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.661586    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"]
Nov 04 01:21:50.750725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.727763    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.750725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.740901    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"]
Nov 04 01:21:50.860586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.860494    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" event={"ID":"4f68d84c-81d2-451e-a0c5-bddcfcc625d5","Type":"ContainerStarted","Data":"3586ce424de1be66ad9c142fe509c22d656f979b0e87d0d647bc3e0ade25a3cc"}
Nov 04 01:21:50.860843 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.860820    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" event={"ID":"4f68d84c-81d2-451e-a0c5-bddcfcc625d5","Type":"ContainerStarted","Data":"63222f5acd8c94fe8b48b8cca6d78ee9a13179101d01baeeff2bcb311206aa15"}
Nov 04 01:21:50.892695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.873685    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" event={"ID":"9640fa9a-743b-4a7c-bc0b-88f2a1299c03","Type":"ContainerStarted","Data":"f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f"}
Nov 04 01:21:50.892695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.873746    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" event={"ID":"9640fa9a-743b-4a7c-bc0b-88f2a1299c03","Type":"ContainerStarted","Data":"2d5a18ddf4fad65272943d676d0cb16ad785c1eacc7a8070f4081af6d998abdd"}
Nov 04 01:21:50.925712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.898903    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:50.925712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.898963    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.957639    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.957720    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.957755    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.957821    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwsg9\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.957930    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.957969    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.958047    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.958265    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:50.959688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:50.958293    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.063580    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086143    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086186    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086252    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwsg9\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086337    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086380    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086436    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086645    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.086697    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.087832    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.088155    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.088449    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.115913    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.163941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.117365    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.245120 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.230123    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwsg9\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.245120 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.245027    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:21:51.286704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.251985    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:21:51.286704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.279469    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:51.286704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.279531    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:51.286704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.279890    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:51.286704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.279927    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:51.366701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.343269    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podStartSLOduration=3.3432456569999998 podStartE2EDuration="3.343245657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:51.047789963 +0000 UTC m=+1479.709930025" watchObservedRunningTime="2026-11-04 01:21:51.343245657 +0000 UTC m=+1480.005385719"
Nov 04 01:21:51.366701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.360486    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.432291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.396269    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.446559 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.446486    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo\") pod \"e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") " pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.590958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.590922    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:51.693742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.689623    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podStartSLOduration=3.689591071 podStartE2EDuration="3.689591071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:51.215997991 +0000 UTC m=+1479.878138055" watchObservedRunningTime="2026-11-04 01:21:51.689591071 +0000 UTC m=+1480.351731133"
Nov 04 01:21:51.697726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.694211    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"]
Nov 04 01:21:51.759706 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:21:51.759640    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d52d14d_2ccb_407f_903b_fcc955a84f8d.slice/crio-4d63370941f6dd5ab0d3a9aa4b0b6de353e5ac45bd6dceb465f8b81feed60e52 WatchSource:0}: Error finding container 4d63370941f6dd5ab0d3a9aa4b0b6de353e5ac45bd6dceb465f8b81feed60e52: Status 404 returned error can't find the container with id 4d63370941f6dd5ab0d3a9aa4b0b6de353e5ac45bd6dceb465f8b81feed60e52
Nov 04 01:21:51.808639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.807792    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:51.808639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.807889    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:51.808639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.807938    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:51.960748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.959198    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:51.960748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:51.959252    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:52.061623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.061581    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" event={"ID":"5d52d14d-2ccb-407f-903b-fcc955a84f8d","Type":"ContainerStarted","Data":"4d63370941f6dd5ab0d3a9aa4b0b6de353e5ac45bd6dceb465f8b81feed60e52"}
Nov 04 01:21:52.098508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.098063    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" event={"ID":"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1","Type":"ContainerStarted","Data":"7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2"}
Nov 04 01:21:52.098508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.098151    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" event={"ID":"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1","Type":"ContainerStarted","Data":"d118fe19ddb0ef5fb0d9f6c473ecc350a78c76601a971980d2e1cb6a4674525b"}
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.267754    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.267864    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.267904    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.268279    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.268327    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:21:52.386249    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.386478    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:52.445775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.386516    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:52.617584 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.617476    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:52.893688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.890153    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:52.893688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:52.890214    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:53.132170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.097507    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podStartSLOduration=4.097486052 podStartE2EDuration="4.097486052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:52.267145686 +0000 UTC m=+1480.929285761" watchObservedRunningTime="2026-11-04 01:21:53.097486052 +0000 UTC m=+1481.759626113"
Nov 04 01:21:53.142694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.132836    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"]
Nov 04 01:21:53.147744 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:21:53.142983    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a73eb28_2cfd_4566_b73e_fedc8d771a1f.slice/crio-719edbc4678035a3da7fa448c23642ca5129aab39f956ba850c617571423b259 WatchSource:0}: Error finding container 719edbc4678035a3da7fa448c23642ca5129aab39f956ba850c617571423b259: Status 404 returned error can't find the container with id 719edbc4678035a3da7fa448c23642ca5129aab39f956ba850c617571423b259
Nov 04 01:21:53.162826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.162764    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:21:53.282765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.274139    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:53.282765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.274198    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:53.282765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.280791    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:53.282765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.280842    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:53.317709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.314276    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:53.317709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.314338    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:53.317709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.314429    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" event={"ID":"5d52d14d-2ccb-407f-903b-fcc955a84f8d","Type":"ContainerStarted","Data":"f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41"}
Nov 04 01:21:53.321414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.321380    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:53.367688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.367594    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:53.367886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.367719    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:53.367886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.367768    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:21:53.385155 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.385107    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:53.484514 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.484475    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:53.660690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.640110    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podStartSLOduration=3.640080914 podStartE2EDuration="3.640080914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:53.500972071 +0000 UTC m=+1482.163112132" watchObservedRunningTime="2026-11-04 01:21:53.640080914 +0000 UTC m=+1482.302220978"
Nov 04 01:21:53.673688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.672761    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:53.673688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.672838    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:53.679795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.679754    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:53.899710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.898252    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:53.899710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.898318    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:53.927076 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.927010    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:53.927972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.927925    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:21:53.949366 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:53.949320    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.344137 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.344056    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.345873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.082833    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.371467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.126416    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:21:54.372025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.165753    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392026    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.287788    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392456    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.287835    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392518    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.290883    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392564    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.000858    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392787    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392854    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:21:54.400714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.392915    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:21:54.402710 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.286787    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.533038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.532644    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.540013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.539973    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:21:54.692757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.681850    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:21:54.692757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.690898    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:21:54.721739 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.721468    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:54.721739 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.721553    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:54.937843 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.937794    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:54.938605 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.938400    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:21:54.942688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:54.938814    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:21:55.108864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.107813    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.135472 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.129704    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:55.151721 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.137987    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:21:55.257754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.249267    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.257754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.249322    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:55.270695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.262039    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.270695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.262116    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:55.286958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.276286    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.286958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.276360    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:55.395834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.395786    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:55.447042 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.410575    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:55.447042 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.444731    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:21:55.447042 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.445545    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:21:55.447042 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.445886    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:21:55.466533 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.464820    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:21:55.484689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.480916    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:55.595788 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.559953    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:55.655689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.646120    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.655689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.646179    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:21:55.655689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.646451    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:55.655689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.646503    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.655689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.646540    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:55.694936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.694885    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:55.702777 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.702732    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" event={"ID":"6a73eb28-2cfd-4566-b73e-fedc8d771a1f","Type":"ContainerStarted","Data":"719edbc4678035a3da7fa448c23642ca5129aab39f956ba850c617571423b259"}
Nov 04 01:21:55.703861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.703839    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:21:55.710688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.704088    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:21:55.710911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.710893    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:21:55.743687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.711068    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:21:55.951694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.945746    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:21:55.951694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.946114    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:21:55.951694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.946887    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:55.951694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:55.946926    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:56.123691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.120978    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:21:56.145695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.144831    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": context deadline exceeded"
Nov 04 01:21:56.258746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.258448    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:56.258746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.258511    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:56.258746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.258589    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:56.258746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.258611    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:56.258746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.258681    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:56.258746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.258701    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:56.460703 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.416134    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.460703 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.417354    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.503733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.468247    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:21:56.503733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.469680    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.503733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.469928    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:21:56.503733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.470148    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:21:56.551752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.530228    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.551752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.551635    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podStartSLOduration=6.551612807 podStartE2EDuration="6.551612807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:21:56.467777436 +0000 UTC m=+1485.129917498" watchObservedRunningTime="2026-11-04 01:21:56.551612807 +0000 UTC m=+1485.213752869"
Nov 04 01:21:56.717048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.623139    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.717048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.626232    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:21:56.717048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.626277    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:21:56.717048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.700949    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:56.717048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.701012    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:56.717048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.701256    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.775703 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.761938    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" event={"ID":"6a73eb28-2cfd-4566-b73e-fedc8d771a1f","Type":"ContainerStarted","Data":"9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52"}
Nov 04 01:21:56.940217 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.940174    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:56.940751 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.940716    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:56.958715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.958106    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:56.962720 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:56.959750    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.137790 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.137746    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:57.281747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.264276    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:57.281747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.264338    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:57.281747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.265636    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:57.281747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.265701    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:57.281747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.265812    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:57.281747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.265838    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:57.425827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.420374    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:57.425827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.421554    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.428785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.428738    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/health\": context deadline exceeded"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.640944    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.641001    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.640767    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.642168    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.642277    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.642488    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.642679    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.642858    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.643042    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.644347    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:21:57.667697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.644377    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:21:57.712233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.711091    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:21:57.716385 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.716211    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:57.953641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.953547    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.044812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.044763    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:58.072030 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.010856    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:58.072030 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:57.983623    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/health\": context deadline exceeded"
Nov 04 01:21:58.072030 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.010896    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:21:58.096424 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.096232    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:58.251745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.248030    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.370033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.317108    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:58.370033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.248893    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.370033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.317229    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:58.370033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.249209    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.370033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.317299    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:58.439693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.425190    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:58.439693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.425598    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": context deadline exceeded"
Nov 04 01:21:58.501941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.501849    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:58.599125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.599083    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.603801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.603762    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:21:58.636686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.636633    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.651793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.651756    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:58.652027 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.646939    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:21:58.652181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.645793    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:21:58.655685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.646938    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:58.660136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.647004    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:21:58.665034 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.647022    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:58.685753 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.646977    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:21:58.741050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.711299    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:58.741050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.728194    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:21:58.889783 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.889749    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 04 01:21:58.890043 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:58.890017    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 04 01:21:59.049813 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.049774    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.050473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.050348    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.238276 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.238185    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:21:59.238559 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.238503    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:21:59.245162 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.245128    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:21:59.245427 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.245392    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:21:59.367361 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.367326    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 04 01:21:59.368039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.368007    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 04 01:21:59.428024 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.427977    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.428619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.428268    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.599022 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.598928    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:21:59.599297 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.599252    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:21:59.634719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.634685    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:21:59.635014 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.634972    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:21:59.655122 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.655064    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:21:59.656981 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.656271    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:21:59.668627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.666751    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.668627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.667076    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 01:21:59.668627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.667322    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:21:59.742860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.742809    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.743479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.743450    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:21:59.943938 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:21:59.943898    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.063448 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.062603    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:22:00.064150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.064032    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.244763 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.244359    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 04 01:22:00.244763 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.244437    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 04 01:22:00.245051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.245000    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:22:00.245127 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.245050    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:22:00.367874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.367819    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Nov 04 01:22:00.428493 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.428435    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": context deadline exceeded"
Nov 04 01:22:00.430169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.429445    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.531927 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.531763    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.598845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.598711    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:00.598845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.598771    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:00.631153 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.631041    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:00.631153 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.631100    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:00.659987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.658768    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.659987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.659631    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.667753 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.666985    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.668599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.668012    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.668599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.668317    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:00.745712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.743782    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:00.745712 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.744218    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:22:00.945798 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.945762    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:00.946048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:00.946022    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.066866 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.066808    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.068368 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.068335    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.236265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.236164    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:22:01.236557 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.236511    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:22:01.405832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.405787    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:01.422853 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.422814    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.451495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.449127    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.451495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.449938    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.608744 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.599564    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:01.608744 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.599626    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:01.642692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.638776    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:01.643021 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.642985    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:01.671694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.671629    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.701763 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.701191    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:01.704306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.701238    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:01.704758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.701260    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 01:22:01.708715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.701331    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:01.747888 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.747828    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:01.748624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.748587    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:01.996913 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.996867    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:01.997232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:01.997184    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.079822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.079778    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:22:02.080870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.080837    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.104882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.104830    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.234116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.234072    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:22:02.234415 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.234386    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:22:02.296863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.296763    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:02.297139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.297108    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.462692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.453392    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:22:02.462692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.453318    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.487692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.470799    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:02.487692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.470883    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.611752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.609864    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:02.611752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.609953    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:02.639912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.639869    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:02.640274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.640186    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:02.703705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.701783    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.703980 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.703944    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.704799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.704747    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.716845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.716786    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 01:22:02.725147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.725101    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:02.748973 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.748907    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:02.750886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.750847    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:02.794381 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:02.794341    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:03.006694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.003795    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.044816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.044773    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:03.045128 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.045082    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.099693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.082822    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:22:03.099693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.083878    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.159099 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.159039    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27" podUID="e83f961f-6e0a-47f8-800b-8b67df7c8c82" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.21:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.237443 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.237392    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 04 01:22:03.237792 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.237739    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 04 01:22:03.346895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.346788    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:03.347177 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.347134    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.461690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.458766    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:22:03.461690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.459633    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.478705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.473746    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:03.478705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.473809    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.599154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.599020    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:03.599154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.599089    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:03.630713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.630060    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:03.630713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.630123    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:03.704051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.704001    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.705175 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.705146    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:03.705482 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.705356    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:03.720822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.720753    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.725776 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.725733    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:03.752799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.752755    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:03.753421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:03.753030    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:04.159689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.113189    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:22:04.159689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.129895    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.175699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.169751    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:22:04.175699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.174513    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:22:04.191724 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.191691    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:22:04.222718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.199359    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:22:04.242499 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.242460    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:04.242822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.242789    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.277913 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.277878    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:22:04.301692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.290545    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:04.501749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.472764    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:22:04.503414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.478403    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.504046 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.503375    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:04.508047 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.508003    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.601620 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.601583    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:04.601900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.601874    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:04.635134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.635085    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:04.635414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.635384    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:04.707760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.706791    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:04.707760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.707203    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.707760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.707502    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.722259 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.721180    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.728484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.727744    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:04.761967 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.761866    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:04.814241 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.773409    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:22:04.896758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.896715    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:22:04.917899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:04.917862    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:22:05.238689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.212303    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.238689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.213094    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.338770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.285600    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:22:05.338770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.324800    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:05.338770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.324876    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.338770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.325237    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:05.338770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.325275    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.492935 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.480966    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.540540 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.540100    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:05.540540 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.540164    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.626706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.608981    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:05.626706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.609044    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:05.628069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.628031    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.691905 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.691851    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:05.692226 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.692182    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:05.710699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.709793    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.710699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.710173    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:05.710699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.710463    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:05.737741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.737697    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.738328 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.738294    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:05.791766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.791592    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:22:05.800692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.791964    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"]
Nov 04 01:22:05.811977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.811946    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"]
Nov 04 01:22:05.812347 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.812329    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.821163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.821121    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:05.918268 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.918219    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.918629 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.918595    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.918883 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.918864    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.919433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.919412    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.919625 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.919594    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.919893 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.919866    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcltv\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.920048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.920030    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.920248 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.920216    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.920456 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.920424    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:05.979159 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:05.979082    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024564    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024635    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024746    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024824    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024870    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024925    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcltv\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.024959    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.025007    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.026089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.025066    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.034396 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.034353    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.035269 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.035246    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.035504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.035203    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.037188 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.037168    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.037628 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.037598    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.040791 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.040742    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.084022 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.083908    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.086499 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.086430    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.109816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.109739    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcltv\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.111697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.111219    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:22:06.182618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.182507    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:06.195233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.194939    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:06.221169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.220774    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.432395 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.432167    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:06.433434 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.433041    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:06.433434 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.433094    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.495716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.480819    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.499861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.496135    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:22:06.602697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.602243    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:22:06.608224 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.608190    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:06.628476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.628419    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:06.629070 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.628340    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:06.629280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.629252    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.631917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.629878    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:06.631917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.629912    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:06.631917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.630363    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:22:06.632828 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.632808    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:06.632977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.632956    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.649263 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.649208    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.658319 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.658270    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:06.721805 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.713015    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.721805 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.713367    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:06.721805 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.713620    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:06.747070 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.747017    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:06.822564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:06.821761    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.115778 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.113769    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.133878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.133777    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.317692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.261908    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.433977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.433766    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/health\": context deadline exceeded"
Nov 04 01:22:07.463280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.462804    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:07.463280 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.462872    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.463735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.463607    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.518730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.486589    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.518730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.492110    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:22:07.518730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.509772    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:22:07.796750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.785723    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.796750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.786133    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:07.796750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.786444    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:07.796750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.788890    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:07.796750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.788943    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:07.816701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.814738    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:22:07.827314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.827257    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.851910    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.829546    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.831847    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.854832    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.854884    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.870812    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:07.883726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:07.870881    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.019651 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.019060    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/health\": context deadline exceeded"
Nov 04 01:22:08.157687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.157187    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.163101 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.163052    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:08.219858 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.219817    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27" podUID="e83f961f-6e0a-47f8-800b-8b67df7c8c82" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.21:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.243945 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.243892    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:22:08.244606 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.244574    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:22:08.244941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.244927    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:22:08.296982 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.296954    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:22:08.413922 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.344867    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.508690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.502016    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.510320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.510281    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.666904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.652847    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:22:08.666904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.652968    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:22:08.666904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.653036    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:22:08.674276 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.674244    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:08.674524 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.674489    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:08.709025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.704624    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:08.709025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.704693    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:08.766438 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.766397    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"]
Nov 04 01:22:08.889690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.885608    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.889967 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.889927    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.933124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.890128    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.958084 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.927274    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.958846 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.927309    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.959526 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.927331    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.960191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.927471    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:08.960352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.960320    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.970631 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.939847    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 01:22:08.984822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.984791    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:08.986312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.985074    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:08.988477 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:08.986733    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:22:09.020047 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.020019    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:22:09.047446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.047344    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:22:09.060912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.060884    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:22:09.109687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.107212    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:09.196111 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.196066    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" event={"ID":"c0c47b89-8d4d-425c-aada-e65032ca072a","Type":"ContainerStarted","Data":"03b6737b394d367375494fa712d43cada505a96a1480ac44eac6f465dd86a790"}
Nov 04 01:22:09.293454 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.293426    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:22:09.486125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.456415    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:09.533549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.514958    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": context deadline exceeded"
Nov 04 01:22:09.533549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.515643    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:09.616796 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.616759    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:09.617067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.617041    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:09.646714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.644157    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:09.646714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.644237    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:09.732148 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.732120    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:22:09.855776 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.855688    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:22:09.944357 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.944312    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": context deadline exceeded"
Nov 04 01:22:09.959724 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.959683    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:10.008701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.969085    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:10.009345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:09.975307    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:10.029690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.017801    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:10.041202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.041165    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:10.048968 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.048926    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:10.049562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.041614    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:22:10.104760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.088947    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:10.105643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.105611    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:22:10.323226 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.323189    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:22:10.346603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.346503    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:22:10.442286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.442258    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:22:10.496698 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.495855    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:22:10.513685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.497604    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:22:10.575461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.575357    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:10.582739 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.582704    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:22:10.649281 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.649250    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 04 01:22:10.653866 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.652744    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 04 01:22:10.683770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.683739    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:22:10.715468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.715413    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Nov 04 01:22:10.715771 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:10.715733    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Nov 04 01:22:11.431793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:11.431749    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" event={"ID":"c0c47b89-8d4d-425c-aada-e65032ca072a","Type":"ContainerStarted","Data":"7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6"}
Nov 04 01:22:11.768750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:11.765028    4434 scope.go:122] "RemoveContainer" containerID="ddfd8e1ec00674808840e2d8742c35c463db986a27885a409e2c6fe50f3ea9e7"
Nov 04 01:22:12.032990 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.032898    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:12.198689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.198158    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:12.200976 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.200927    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:22:12.201387 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.201356    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 01:22:12.321743 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.321057    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:22:12.325181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.325052    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:22:12.448581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.448522    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podStartSLOduration=7.448504972 podStartE2EDuration="7.448504972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:22:11.60468506 +0000 UTC m=+1500.266825126" watchObservedRunningTime="2026-11-04 01:22:12.448504972 +0000 UTC m=+1501.110645034"
Nov 04 01:22:12.683195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.682797    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:12.683195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:12.682868    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:13.160150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:13.160117    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:22:13.161213 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:13.161192    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:22:13.201689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:13.201002    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:22:13.201689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:13.201072    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 01:22:13.703768 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:13.703719    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:14.195907 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:14.195869    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:22:14.196329 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:14.195929    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 01:22:14.448728 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:14.448588    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:15.162821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:15.162777    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:15.196327 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:15.196278    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:22:15.196817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:15.196355    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 01:22:15.996647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:15.996610    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:16.197055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:16.197017    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:22:16.197608 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:16.197580    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 01:22:17.195850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:17.195809    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 04 01:22:17.196024 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:17.195864    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 04 01:22:18.201257 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:18.201211    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:18.201655 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:18.201502    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:18.202284 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:18.202262    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:22:18.354051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:18.354019    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:18.585533 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:18.585451    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:18.776432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:18.776401    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:20.259068 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:20.259035    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:20.476579 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:20.476545    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:21.200940 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:21.200904    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:21.898840 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:22:21.898800    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:22:22.048060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:22.048024    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:23.298326 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:23.298292    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:23.534932 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:23.534904    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:23.737261 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:23.737225    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:24.232918 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:24.232885    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:24.418227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:24.418196    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:26.067822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:26.067792    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:26.252778 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:26.252739    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:27.416542 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:27.416508    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:27.598300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:27.598268    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:27.779825 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:27.779743    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:29.411301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:29.411250    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:29.604652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:29.604618    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:31.227078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:31.227046    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:32.707320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:32.707285    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:32.891416 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:32.891363    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:33.273633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:33.273596    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:34.746235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:34.746195    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:34.963200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:34.963140    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:34.990242 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:34.990211    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"]
Nov 04 01:22:34.994197 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:34.994174    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.066987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.066954    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067257 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067235    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067287    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s867\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067521 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067493    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067657    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067735    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067789    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067885 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067871    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.067928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.067902    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.117842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.117806    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"]
Nov 04 01:22:35.169497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169455    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.169711 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169519    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s867\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.169795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169706    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.169857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169816    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.169914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169881    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.169914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169906    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.170023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169934    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.170023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.169966    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.170139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.170017    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.170139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.170037    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.170447 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.170423    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.170793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.170770    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.171092 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.171069    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.171347 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.171324    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.173584 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.173557    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.178792 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.174320    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.236315 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.235694    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s867\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.236315 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.236044    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token\") pod \"e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") " pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.307915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.307839    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:35.754589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.754556    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"]
Nov 04 01:22:35.796144 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:22:35.796105    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf17bf0ce_3d87_457d_ac9c_b27d5ddcb930.slice/crio-115cea4c8b3e443dfe429e7024a8722f123c32e6d98c8c0ee4f133dad1eac0be WatchSource:0}: Error finding container 115cea4c8b3e443dfe429e7024a8722f123c32e6d98c8c0ee4f133dad1eac0be: Status 404 returned error can't find the container with id 115cea4c8b3e443dfe429e7024a8722f123c32e6d98c8c0ee4f133dad1eac0be
Nov 04 01:22:35.798899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.798865    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:22:35.798987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.798938    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:22:35.798987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:35.798968    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:22:36.646257 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:36.646217    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" event={"ID":"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930","Type":"ContainerStarted","Data":"dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3"}
Nov 04 01:22:36.646474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:36.646318    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" event={"ID":"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930","Type":"ContainerStarted","Data":"115cea4c8b3e443dfe429e7024a8722f123c32e6d98c8c0ee4f133dad1eac0be"}
Nov 04 01:22:36.792693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:36.791975    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podStartSLOduration=2.791954984 podStartE2EDuration="2.791954984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:22:36.779797279 +0000 UTC m=+1525.441937355" watchObservedRunningTime="2026-11-04 01:22:36.791954984 +0000 UTC m=+1525.454095047"
Nov 04 01:22:37.318685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:37.318028    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:37.333900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:37.333834    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:37.334295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:37.334242    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:37.687688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:37.684218    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:38.056740 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.054105    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:38.382688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.381399    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:38.382688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.381477    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:38.430689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.418774    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.490690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.489754    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"]
Nov 04 01:22:38.516735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.516245    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:38.520693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.517081    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.575692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.531729    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:38.607761 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.539520    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.622694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.574113    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.622694 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.575966    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.688756 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.635953    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:38.720059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.712157    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-b45d1fe0-dockercfg-zh2mz\""
Nov 04 01:22:38.720059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.713074    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-b45d1fe0\""
Nov 04 01:22:38.855381 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.728428    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"]
Nov 04 01:22:38.855381 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.789738    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.855381 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.814914    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.905703 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.885640    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js8td\" (UniqueName: \"kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td\") pod \"payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") " pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:38.905703 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.885849    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") " pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:38.978750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.973735    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"]
Nov 04 01:22:38.978750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.973920    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:38.993688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:38.990546    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:22:39.054972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.054922    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") " pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:39.057282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.055391    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:39.080589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.080555    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js8td\" (UniqueName: \"kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td\") pod \"payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") " pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:39.135195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.135161    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume\") pod \"payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") " pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:39.224706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.217879    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"]
Nov 04 01:22:39.224706 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.218024    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.274704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.264074    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9xz\" (UniqueName: \"kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz\") pod \"payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.274704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.264254    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.357086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.357050    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js8td\" (UniqueName: \"kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td\") pod \"payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") " pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:39.357457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.357416    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:39.357658 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.357621    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:39.401118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.379038    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9xz\" (UniqueName: \"kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz\") pod \"payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.401118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.379224    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.401118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.380443    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.451898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.426076    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:39.451898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.449590    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:22:39.571744 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.555924    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:39.571744 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.571336    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:39.621970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.598438    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9xz\" (UniqueName: \"kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz\") pod \"payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:39.621970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.598592    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:39.621970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.610270    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:39.621970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.619056    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:39.621970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.619328    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:39.826047 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.825789    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:39.830853 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.830799    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:39.916653 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.916618    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:22:39.925471 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.925440    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:22:39.925769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.925750    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:22:39.925917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.925904    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:22:39.926046 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.926034    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:22:39.926181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.926168    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:22:39.926307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.926295    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:22:39.956870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.956822    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": context deadline exceeded"
Nov 04 01:22:39.977748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.974290    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.013691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:39.984859    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:22:40.057813 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.057767    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.096984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.069144    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:22:40.133817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.133734    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:40.140815 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.140764    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:40.146804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.146761    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:40.157697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.147412    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:22:40.296692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.261782    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:40.296692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.284222    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:40.458920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.341397    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:40.550692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.509001    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:40.550692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.465291    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": context deadline exceeded"
Nov 04 01:22:40.550692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.490467    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:40.550692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.459200    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:40.700847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.632175    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": context deadline exceeded"
Nov 04 01:22:40.700847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.632280    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:40.700847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.632625    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.700847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.642274    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.700847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.643135    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.700847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.643433    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.854746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.843595    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:40.859713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.854858    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:40.994023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:40.993959    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.082931 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.080824    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.160734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.153270    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:41.160734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.153449    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Nov 04 01:22:41.245750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.234551    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:41.259976 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.259829    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.341853 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.341805    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:41.505877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.505762    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:41.531851 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.527865    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.571093 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.571047    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:41.571370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.571341    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.634823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.634771    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:41.643811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.643768    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": context deadline exceeded"
Nov 04 01:22:41.734412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.668970    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Nov 04 01:22:41.739690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.669015    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:41.767764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.763571    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.796884 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.774988    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:41.859832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.859788    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:41.889724 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:41.889684    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:42.063760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.038482    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:42.260843 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.163766    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:42.261445 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.174802    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:42.262801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.175071    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:22:42.427778 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.427730    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:42.431726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.428058    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:42.497908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.497871    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:42.572691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.564922    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:42.636793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.636748    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:42.673692 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:22:42.670552    4434 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.193s"
Nov 04 01:22:43.010704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.009006    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"]
Nov 04 01:22:43.010704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.744857    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.762233    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.707003    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": context deadline exceeded"
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.782326    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.839907    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.036589    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.008813    4434 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.048707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.950851    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:43.121765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.950915    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": context deadline exceeded"
Nov 04 01:22:43.258014 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:42.840692    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:43.267305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.267265    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.327729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.305179    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.328082 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.328048    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.356515 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.329843    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 04 01:22:43.383785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.383307    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:43.383785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.383366    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:43.464965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.441815    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88b27" podUID="e83f961f-6e0a-47f8-800b-8b67df7c8c82" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.21:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.464965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.464801    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:43.464965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.464866    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.467289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.467259    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:43.703391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.703351    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.725182 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.725145    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:43.725466 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.725437    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.801724 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.703245    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:43.933592 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.903922    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f" event={"ID":"14420692-cad5-4afe-b05a-ac82e05af453","Type":"ContainerStarted","Data":"1f7489c300530931744cc8484895f9d6ac4059d0e9964f40d1ea9cc3c63b1be6"}
Nov 04 01:22:43.933592 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:43.904012    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"]
Nov 04 01:22:44.016936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.009709    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:44.053334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.048516    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:22:44.053334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.048878    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8000/ready\": context deadline exceeded"
Nov 04 01:22:44.053334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.049222    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 04 01:22:44.053334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.049498    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Nov 04 01:22:44.159692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.156346    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:44.331729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.321038    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 04 01:22:44.331729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.322102    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:44.331729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.322145    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:44.351697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.349131    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:44.358460 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.357743    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:44.504196 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.504165    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:22:44.547831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.547785    4434 patch_prober.go:29] interesting pod/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:44.548071 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.548041    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:44.552991 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.552967    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:22:44.823379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.823324    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Nov 04 01:22:44.956962 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.956495    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:22:44.958557 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.957392    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:22:44.996658 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:44.996607    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:22:45.014467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.013549    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:45.034417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.034382    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:22:45.037520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.037498    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:22:45.038889 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.038861    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv" event={"ID":"f92b7329-d2bd-4aec-bda1-93be73515f28","Type":"ContainerStarted","Data":"3de91e7a2e84d5177194aeb7cadd9f45e421b52bd63a98d26e7a1aa70f3fff5e"}
Nov 04 01:22:45.039013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.038927    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:45.039519 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.039501    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:22:45.053686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.049446    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:45.053686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.050109    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:45.121544 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.119716    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f" event={"ID":"14420692-cad5-4afe-b05a-ac82e05af453","Type":"ContainerStarted","Data":"b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac"}
Nov 04 01:22:45.121544 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.120246    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:45.121544 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.120596    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:22:45.124115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.124081    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:22:45.126442 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.125899    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:22:45.218582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.218482    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv" podStartSLOduration=7.218467474 podStartE2EDuration="7.218467474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:22:45.217170788 +0000 UTC m=+1533.879310851" watchObservedRunningTime="2026-11-04 01:22:45.218467474 +0000 UTC m=+1533.880607534"
Nov 04 01:22:45.308351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.308312    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:45.308540 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:45.308370    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:46.134674 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:46.134616    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv" event={"ID":"f92b7329-d2bd-4aec-bda1-93be73515f28","Type":"ContainerStarted","Data":"1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab"}
Nov 04 01:22:46.253139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:46.253086    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f" podStartSLOduration=8.253071158000001 podStartE2EDuration="8.253071158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:22:46.250449239 +0000 UTC m=+1534.912589300" watchObservedRunningTime="2026-11-04 01:22:46.253071158 +0000 UTC m=+1534.915211246"
Nov 04 01:22:46.308697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:46.308650    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:46.308863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:46.308718    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:47.283757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:47.283647    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:47.308904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:47.308864    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 04 01:22:47.309073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:47.308932    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 04 01:22:47.474299 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:47.474258    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:47.654367 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:47.654325    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:49.085841 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.085796    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:49.279809 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.279778    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:49.309143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.309100    4434 patch_prober.go:29] interesting pod/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 04 01:22:49.309323 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.309191    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:22:49.513742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.513706    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:49.514012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.513993    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:49.514756 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:49.514739    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:22:51.900166 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:22:51.900127    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:22:52.909609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:52.909577    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:54.192742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:54.192701    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:22:56.138228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:56.138198    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:22:57.147742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:57.147714    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:22:58.021293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:22:58.021258    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:03.437993 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:03.437922    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:03.921048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:03.921006    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:04.270186 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.270099    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"]
Nov 04 01:23:04.275077 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.275053    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.352390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352344    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.352566 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352416    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.352566 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352549    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.352731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352654    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.352801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352779    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.352847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352827    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.353026 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.352999    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.353080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.353057    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.353140 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.353124    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.388186 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.388130    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"]
Nov 04 01:23:04.454186 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454148    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454221    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454248    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454307    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454355    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454402    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454434    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454604    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.454685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454624    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.455117 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454755    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.455117 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454780    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.455117 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454865    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.455117 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.454899    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.455454 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.455432    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.456725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.456703    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.456953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.456935    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.514594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.514558    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.516761 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.516729    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token\") pod \"e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") " pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:04.591513 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:04.591182    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:05.044409 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.044381    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"]
Nov 04 01:23:05.046330 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:23:05.046306    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c9381e_e7b1_4579_8b32_d32ad9add16e.slice/crio-52f1147c3a6cccc31cd898b4b3494e31bda018e7aff37479736c412dcca75ccd WatchSource:0}: Error finding container 52f1147c3a6cccc31cd898b4b3494e31bda018e7aff37479736c412dcca75ccd: Status 404 returned error can't find the container with id 52f1147c3a6cccc31cd898b4b3494e31bda018e7aff37479736c412dcca75ccd
Nov 04 01:23:05.049034 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.049001    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:05.049121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.049067    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:05.049121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.049098    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:05.280994 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.280954    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" event={"ID":"04c9381e-e7b1-4579-8b32-d32ad9add16e","Type":"ContainerStarted","Data":"2e6566f88d286468085f542e9bf52f405d4b1d67acbced7394431557ea9f1006"}
Nov 04 01:23:05.281423 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.281404    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" event={"ID":"04c9381e-e7b1-4579-8b32-d32ad9add16e","Type":"ContainerStarted","Data":"52f1147c3a6cccc31cd898b4b3494e31bda018e7aff37479736c412dcca75ccd"}
Nov 04 01:23:05.426320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.426258    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podStartSLOduration=1.42624115 podStartE2EDuration="1.42624115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:05.421259734 +0000 UTC m=+1554.083399795" watchObservedRunningTime="2026-11-04 01:23:05.42624115 +0000 UTC m=+1554.088381210"
Nov 04 01:23:05.597826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.597786    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:05.648484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.648452    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"]
Nov 04 01:23:05.652353 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.652336    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.673917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.673732    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:05.712511 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.712434    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"]
Nov 04 01:23:05.793784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.793747    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794074    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqs2x\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794121    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794242    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794274    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794414 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794295    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794471 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794444    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794547    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.794625 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.794583    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897008 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.896974    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897064    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897097    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897123    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897219    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqs2x\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897274    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897366    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897427    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897517    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897512    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897571    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897759 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897699    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.897867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.897843    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.898079 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.898058    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.899627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.899599    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.899761 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.899718    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.959614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.959581    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqs2x\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.959614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.959607    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token\") pod \"e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") " pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:05.991840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.991749    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"]
Nov 04 01:23:05.993912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.993878    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:06.000202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:05.999846    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.087167    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"]
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.099862    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.099971    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100003    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100040    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100109    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100132    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100151    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100182    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.104432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.100207    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202178    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202247    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202312    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202408    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202444    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202477    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202529    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202565    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202759    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.202898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202780    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.203025 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.202997    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.203157 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.203133    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.203325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.203298    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.203654 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.203622    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.205490 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.205467    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.205832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.205815    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.260833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.260772    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.260833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.260782    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96\") pod \"e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") " pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.321369 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.321338    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:06.435832 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:23:06.435790    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc16b56ea_40c6_4e0a_a895_67422da97231.slice/crio-e98e73a7e299e03b2261dbb101dffeee48ff39c49f8387d03b3b437603569e6d WatchSource:0}: Error finding container e98e73a7e299e03b2261dbb101dffeee48ff39c49f8387d03b3b437603569e6d: Status 404 returned error can't find the container with id e98e73a7e299e03b2261dbb101dffeee48ff39c49f8387d03b3b437603569e6d
Nov 04 01:23:06.435986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.435871    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"]
Nov 04 01:23:06.439144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.439111    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:06.439422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.439399    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:06.439559 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.439543    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:06.596531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.596495    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:23:06.596740 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.596543    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:23:06.743513 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.743485    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"]
Nov 04 01:23:06.746036 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:23:06.746008    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17bbbd33_37ab_49ba_944e_325ea2f98749.slice/crio-6607ac15a7872b49935f86f4a66b996f9a60b065516902667b498b9c9b627788 WatchSource:0}: Error finding container 6607ac15a7872b49935f86f4a66b996f9a60b065516902667b498b9c9b627788: Status 404 returned error can't find the container with id 6607ac15a7872b49935f86f4a66b996f9a60b065516902667b498b9c9b627788
Nov 04 01:23:06.749022 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.748991    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:06.749110 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.749056    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:06.749110 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:06.749089    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:07.321758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.318641    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" event={"ID":"c16b56ea-40c6-4e0a-a895-67422da97231","Type":"ContainerStarted","Data":"429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730"}
Nov 04 01:23:07.321758 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.318729    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" event={"ID":"c16b56ea-40c6-4e0a-a895-67422da97231","Type":"ContainerStarted","Data":"e98e73a7e299e03b2261dbb101dffeee48ff39c49f8387d03b3b437603569e6d"}
Nov 04 01:23:07.324769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.324729    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" event={"ID":"17bbbd33-37ab-49ba-944e-325ea2f98749","Type":"ContainerStarted","Data":"9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4"}
Nov 04 01:23:07.324928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.324785    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" event={"ID":"17bbbd33-37ab-49ba-944e-325ea2f98749","Type":"ContainerStarted","Data":"6607ac15a7872b49935f86f4a66b996f9a60b065516902667b498b9c9b627788"}
Nov 04 01:23:07.461504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.461448    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podStartSLOduration=2.4614289400000002 podStartE2EDuration="2.46142894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:07.458419728 +0000 UTC m=+1556.120559786" watchObservedRunningTime="2026-11-04 01:23:07.46142894 +0000 UTC m=+1556.123569007"
Nov 04 01:23:07.592364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.592172    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:23:07.592364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.592251    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:23:07.613687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.603003    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podStartSLOduration=2.602981131 podStartE2EDuration="2.602981131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:07.601043611 +0000 UTC m=+1556.263183674" watchObservedRunningTime="2026-11-04 01:23:07.602981131 +0000 UTC m=+1556.265121193"
Nov 04 01:23:07.996689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.995825    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:08.000686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.997654    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:23:08.000686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:07.997927    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:23:08.327937 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:08.327841    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:08.339689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:08.337738    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:23:08.339689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:08.337820    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 01:23:08.662840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:08.662800    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:23:08.687786 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:08.687744    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:23:08.693718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:08.685161    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:09.013331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.013230    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:23:09.014795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.014757    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:23:09.095699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.081910    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:09.334749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.331051    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:23:09.334749 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.331142    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 01:23:09.592215 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.592117    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:23:09.592475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:09.592444    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:23:10.000365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:10.000324    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:23:10.000641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:10.000615    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:23:10.328747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:10.328050    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:23:10.328747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:10.328143    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 01:23:10.596607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:10.596114    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:23:10.596607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:10.596177    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:23:11.000637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.000599    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:23:11.000940 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.000910    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:23:11.322989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.322182    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:23:11.322989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.322239    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 01:23:11.592296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.592204    4434 patch_prober.go:29] interesting pod/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 04 01:23:11.592890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.592852    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 04 01:23:11.994453 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.994413    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:23:11.994767 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:11.994736    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:23:12.322090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:12.321991    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:23:12.322090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:12.322060    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 01:23:12.999687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:12.999143    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 04 01:23:12.999687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:12.999206    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 04 01:23:13.154890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.154845    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:13.155421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.155396    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:13.156298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.156282    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:23:13.322307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.322222    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 04 01:23:13.322307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.322288    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 04 01:23:13.407149 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.407108    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"]
Nov 04 01:23:13.412521 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.412494    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.481425 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.481390    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e400a3a6-serving-cert\""
Nov 04 01:23:13.487643 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.487610    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"]
Nov 04 01:23:13.535934 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.535896    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.536362 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.536331    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.536497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.536476    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.536568 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.536540    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2spz5\" (UniqueName: \"kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.635416 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.635354    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:13.638594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.638560    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2spz5\" (UniqueName: \"kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.638858 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.638833    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.638970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.638952    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.639114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.639096    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.639189 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:23:13.639103    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-serving-cert: secret "maas-api-e2e-ait-e400a3a6-serving-cert" not found
Nov 04 01:23:13.639249 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:23:13.639223    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls podName:a2d6f858-1c1d-4eac-a467-df098059011e nodeName:}" failed. No retries permitted until 2026-11-04 01:23:14.13920239 +0000 UTC m=+1562.801342453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls") pod "maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" (UID: "a2d6f858-1c1d-4eac-a467-df098059011e") : secret "maas-api-e2e-ait-e400a3a6-serving-cert" not found
Nov 04 01:23:13.639989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.639961    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.645311 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.645286    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.698275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.698246    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2spz5\" (UniqueName: \"kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:13.819121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:13.819071    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:14.091932 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.091630    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:14.150840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.150773    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:14.173388 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.173359    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\") pod \"maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:14.327451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.327413    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:14.373795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.372622    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:14.373795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.373739    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:14.375128 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.375083    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:23:14.631588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.631560    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:14.632022 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.631998    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:14.633032 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.633009    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:23:14.762497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.762470    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"]
Nov 04 01:23:14.766231 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:14.766198    4434 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:23:15.441028 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:15.440991    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" event={"ID":"a2d6f858-1c1d-4eac-a467-df098059011e","Type":"ContainerStarted","Data":"c7832b7af5b92eab67d1e2bb9f05dbbd6bb26712b410b520801ce3cb7553438a"}
Nov 04 01:23:16.449613 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:16.449576    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" event={"ID":"a2d6f858-1c1d-4eac-a467-df098059011e","Type":"ContainerStarted","Data":"86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a"}
Nov 04 01:23:16.450138 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:16.449715    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:16.568748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:16.568658    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" podStartSLOduration=2.653504785 podStartE2EDuration="3.568638846s" totalImagesPullingTime="915.134061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:16.566058049 +0000 UTC m=+1565.228198112" watchObservedRunningTime="2026-11-04 01:23:16.568638846 +0000 UTC m=+1565.230778911"
Nov 04 01:23:18.741338 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:18.741276    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:18.902942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:18.902908    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:19.104979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:19.104897    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:21.898156 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:23:21.898116    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:23:22.464334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:22.464305    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:23:23.586764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:23.586649    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:27.336924 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.336891    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"]
Nov 04 01:23:27.341103 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.341080    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469246    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469453 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469296    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469501 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469463    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469585    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hpdq\" (UniqueName: \"kubernetes.io/projected/83e28f11-e50b-4523-b8a5-4647d841c30c-kube-api-access-4hpdq\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469793    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469828    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.469908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.469886    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.470019 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.470006    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e28f11-e50b-4523-b8a5-4647d841c30c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.470057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.470037    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.551545 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.551515    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"]
Nov 04 01:23:27.571150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571108    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hpdq\" (UniqueName: \"kubernetes.io/projected/83e28f11-e50b-4523-b8a5-4647d841c30c-kube-api-access-4hpdq\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571166    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571204    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571251    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571494 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571378    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e28f11-e50b-4523-b8a5-4647d841c30c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571552 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571498    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571724 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571702    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571764    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571770    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.571929 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.571856    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.572163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.572144    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.572305 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.572156    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83e28f11-e50b-4523-b8a5-4647d841c30c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.572422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.572300    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.572422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.572325    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.574324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.574300    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.574730 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.574707    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.629300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.629276    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83e28f11-e50b-4523-b8a5-4647d841c30c-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.629439 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.629358    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hpdq\" (UniqueName: \"kubernetes.io/projected/83e28f11-e50b-4523-b8a5-4647d841c30c-kube-api-access-4hpdq\") pod \"maas-default-gateway-openshift-default-694db55c87-j72g2\" (UID: \"83e28f11-e50b-4523-b8a5-4647d841c30c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:27.655347 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:27.655326    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:28.088586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.088547    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"]
Nov 04 01:23:28.090326 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:23:28.090299    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83e28f11_e50b_4523_b8a5_4647d841c30c.slice/crio-046f1ed972ac06e769242e95fb4155073a006e257c0f93a8aeac385f11e5b1a0 WatchSource:0}: Error finding container 046f1ed972ac06e769242e95fb4155073a006e257c0f93a8aeac385f11e5b1a0: Status 404 returned error can't find the container with id 046f1ed972ac06e769242e95fb4155073a006e257c0f93a8aeac385f11e5b1a0
Nov 04 01:23:28.092925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.092892    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:28.093018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.092956    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:28.093018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.092987    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:28.528489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.528457    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" event={"ID":"83e28f11-e50b-4523-b8a5-4647d841c30c","Type":"ContainerStarted","Data":"46d78483b52f0b2282e5669bc8fd5a3910d0ad66112c896fca581ead730836ce"}
Nov 04 01:23:28.528942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.528511    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" event={"ID":"83e28f11-e50b-4523-b8a5-4647d841c30c","Type":"ContainerStarted","Data":"046f1ed972ac06e769242e95fb4155073a006e257c0f93a8aeac385f11e5b1a0"}
Nov 04 01:23:28.627567 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.627531    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:28.659812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.658173    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:28.663554 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:28.663475    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podStartSLOduration=1.66345644 podStartE2EDuration="1.66345644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:28.658896977 +0000 UTC m=+1577.321037051" watchObservedRunningTime="2026-11-04 01:23:28.66345644 +0000 UTC m=+1577.325596503"
Nov 04 01:23:29.657082 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:29.657051    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:29.657467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:29.657107    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:30.656641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:30.656602    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:30.656807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:30.656656    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:31.655641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:31.655610    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:31.656097 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:31.655708    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:32.657688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:32.656941    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:32.657688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:32.656996    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:33.674687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:33.663035    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:33.674687 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:33.663121    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:34.124793 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:34.123344    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:34.656120 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:34.656076    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:34.656324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:34.656140    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:35.655653 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:35.655619    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-j72g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Nov 04 01:23:35.656186 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:35.655710    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2" podUID="83e28f11-e50b-4523-b8a5-4647d841c30c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Nov 04 01:23:36.750022 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:36.749987    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:36.750627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:36.750501    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:36.751415 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:36.751398    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-j72g2"
Nov 04 01:23:39.882476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:39.882436    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:42.450121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.450088    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"]
Nov 04 01:23:42.455623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.455597    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524486 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524452    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/179fdcce-5744-45d1-ad03-437346736c4d-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524630 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524497    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524630 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524611    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/179fdcce-5744-45d1-ad03-437346736c4d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524719    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524773 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524765    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524786    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524815    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/179fdcce-5744-45d1-ad03-437346736c4d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524950 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524936    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.524985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.524959    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz9rx\" (UniqueName: \"kubernetes.io/projected/179fdcce-5744-45d1-ad03-437346736c4d-kube-api-access-mz9rx\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626378    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/179fdcce-5744-45d1-ad03-437346736c4d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626483    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626535    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626565    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626607    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/179fdcce-5744-45d1-ad03-437346736c4d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626742    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.626827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626771    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz9rx\" (UniqueName: \"kubernetes.io/projected/179fdcce-5744-45d1-ad03-437346736c4d-kube-api-access-mz9rx\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626934    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/179fdcce-5744-45d1-ad03-437346736c4d-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626959    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-workload-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.626984    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.627032    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-workload-certs\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.627131    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-credential-socket\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627502 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.627266    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/179fdcce-5744-45d1-ad03-437346736c4d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.627502 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.627313    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-istio-data\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.629018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.628996    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/179fdcce-5744-45d1-ad03-437346736c4d-istio-envoy\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.629258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.629237    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/179fdcce-5744-45d1-ad03-437346736c4d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.658194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.658159    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"]
Nov 04 01:23:42.684870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.684844    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/179fdcce-5744-45d1-ad03-437346736c4d-istio-token\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.685259 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.685236    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz9rx\" (UniqueName: \"kubernetes.io/projected/179fdcce-5744-45d1-ad03-437346736c4d-kube-api-access-mz9rx\") pod \"maas-default-gateway-openshift-default-694db55c87-2sl8p\" (UID: \"179fdcce-5744-45d1-ad03-437346736c4d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:42.770271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:42.770175    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:43.209440 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.209411    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"]
Nov 04 01:23:43.210948 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:23:43.210921    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179fdcce_5744_45d1_ad03_437346736c4d.slice/crio-4db609162a9c04d408e6dd3161411ac9de7b0891aab9ba26747eb9dc96d9598e WatchSource:0}: Error finding container 4db609162a9c04d408e6dd3161411ac9de7b0891aab9ba26747eb9dc96d9598e: Status 404 returned error can't find the container with id 4db609162a9c04d408e6dd3161411ac9de7b0891aab9ba26747eb9dc96d9598e
Nov 04 01:23:43.213739 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.213708    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:43.213822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.213775    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:43.213822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.213804    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:23:43.627557 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.627520    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" event={"ID":"179fdcce-5744-45d1-ad03-437346736c4d","Type":"ContainerStarted","Data":"aad441213affe62d667dc59f8db652ef7e10404e4313239f6f59149584a03ea9"}
Nov 04 01:23:43.627984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.627580    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" event={"ID":"179fdcce-5744-45d1-ad03-437346736c4d","Type":"ContainerStarted","Data":"4db609162a9c04d408e6dd3161411ac9de7b0891aab9ba26747eb9dc96d9598e"}
Nov 04 01:23:43.770346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.770320    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:43.848678 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:43.848601    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podStartSLOduration=1.848588163 podStartE2EDuration="1.848588163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:23:43.798017674 +0000 UTC m=+1592.460157735" watchObservedRunningTime="2026-11-04 01:23:43.848588163 +0000 UTC m=+1592.510728285"
Nov 04 01:23:44.716284 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:44.716237    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:44.790737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:44.785805    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:44.790737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:44.785875    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:45.788084 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:45.788040    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:45.788723 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:45.788685    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:46.784690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:46.784026    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:46.784690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:46.784103    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:47.781069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:47.781031    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:47.781649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:47.781620    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:48.788719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:48.785409    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:48.788719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:48.785466    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:49.771287 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:49.771249    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:49.771491 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:49.771306    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:50.076728 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:50.076624    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:23:50.771334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:50.771295    4434 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-694db55c87-2sl8p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Nov 04 01:23:50.771508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:50.771390    4434 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p" podUID="179fdcce-5744-45d1-ad03-437346736c4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Nov 04 01:23:51.902143 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:23:51.902103    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:23:51.958750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:51.958718    4434 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:51.959295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:51.959271    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:51.960045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:51.960026    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-694db55c87-2sl8p"
Nov 04 01:23:55.089224 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:23:55.089178    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:00.104527 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:00.104492    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:04.171102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:04.171027    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:05.122901 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:05.122806    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:10.385657 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:10.385613    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:16.026520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:16.026478    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:18.099704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:18.099653    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:18.355322 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:18.355229    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:21.844054 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:21.844020    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:21.900040 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:24:21.900008    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:24:22.493256 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:22.484989    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:27.402517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:27.402482    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:28.373115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:28.373061    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:28.749678 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:28.749497    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:29.770777 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:29.770728    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:32.686708 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:32.686675    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:32.886144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:32.886054    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:34.589941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:34.589911    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:36.391964 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:36.391923    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"]
Nov 04 01:24:36.509258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:36.509227    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"]
Nov 04 01:24:36.804497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:36.804418    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:36.975293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:36.975251    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:37.995698 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:37.992741    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:40.752899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:40.752860    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:41.392288 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.392253    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f" podUID="14420692-cad5-4afe-b05a-ac82e05af453" containerName="payload-processing" containerID="cri-o://b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac" gracePeriod=25
Nov 04 01:24:41.510332 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.510298    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv" podUID="f92b7329-d2bd-4aec-bda1-93be73515f28" containerName="payload-pre-processing" containerID="cri-o://1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab" gracePeriod=25
Nov 04 01:24:41.681239 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.681212    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:24:41.781594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.781558    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td\" (UniqueName: \"kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td\") pod \"14420692-cad5-4afe-b05a-ac82e05af453\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") "
Nov 04 01:24:41.782075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.781692    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume\") pod \"14420692-cad5-4afe-b05a-ac82e05af453\" (UID: \"14420692-cad5-4afe-b05a-ac82e05af453\") "
Nov 04 01:24:41.782143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.782113    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume" pod "14420692-cad5-4afe-b05a-ac82e05af453" (UID: "14420692-cad5-4afe-b05a-ac82e05af453"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:24:41.782464 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.782443    4434 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14420692-cad5-4afe-b05a-ac82e05af453-plugins-config-volume\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:24:41.783690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.783649    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td" pod "14420692-cad5-4afe-b05a-ac82e05af453" (UID: "14420692-cad5-4afe-b05a-ac82e05af453"). InnerVolumeSpecName "kube-api-access-js8td". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:24:41.810362 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.810325    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:24:41.883129 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.883091    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz\" (UniqueName: \"kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz\") pod \"f92b7329-d2bd-4aec-bda1-93be73515f28\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") "
Nov 04 01:24:41.883311 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.883153    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume\") pod \"f92b7329-d2bd-4aec-bda1-93be73515f28\" (UID: \"f92b7329-d2bd-4aec-bda1-93be73515f28\") "
Nov 04 01:24:41.883539 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.883515    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume" pod "f92b7329-d2bd-4aec-bda1-93be73515f28" (UID: "f92b7329-d2bd-4aec-bda1-93be73515f28"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:24:41.884032 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.884009    4434 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f92b7329-d2bd-4aec-bda1-93be73515f28-plugins-config-volume\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:24:41.884032 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.884032    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-js8td\" (UniqueName: \"kubernetes.io/projected/14420692-cad5-4afe-b05a-ac82e05af453-kube-api-access-js8td\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:24:41.885321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.885298    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz" pod "f92b7329-d2bd-4aec-bda1-93be73515f28" (UID: "f92b7329-d2bd-4aec-bda1-93be73515f28"). InnerVolumeSpecName "kube-api-access-wl9xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:24:41.986086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:41.986003    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl9xz\" (UniqueName: \"kubernetes.io/projected/f92b7329-d2bd-4aec-bda1-93be73515f28-kube-api-access-wl9xz\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:24:42.057866 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.057829    4434 generic.go:356] "Generic (PLEG): container finished" podID="14420692-cad5-4afe-b05a-ac82e05af453" containerID="b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac" exitCode=0
Nov 04 01:24:42.058060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.057881    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"
Nov 04 01:24:42.058060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.057911    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f" event={"ID":"14420692-cad5-4afe-b05a-ac82e05af453","Type":"ContainerDied","Data":"b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac"}
Nov 04 01:24:42.058060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.057963    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f" event={"ID":"14420692-cad5-4afe-b05a-ac82e05af453","Type":"ContainerDied","Data":"1f7489c300530931744cc8484895f9d6ac4059d0e9964f40d1ea9cc3c63b1be6"}
Nov 04 01:24:42.058060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.057986    4434 scope.go:122] "RemoveContainer" containerID="b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac"
Nov 04 01:24:42.059354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.059330    4434 generic.go:356] "Generic (PLEG): container finished" podID="f92b7329-d2bd-4aec-bda1-93be73515f28" containerID="1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab" exitCode=0
Nov 04 01:24:42.059479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.059370    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"
Nov 04 01:24:42.059479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.059411    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv" event={"ID":"f92b7329-d2bd-4aec-bda1-93be73515f28","Type":"ContainerDied","Data":"1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab"}
Nov 04 01:24:42.059479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.059455    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv" event={"ID":"f92b7329-d2bd-4aec-bda1-93be73515f28","Type":"ContainerDied","Data":"3de91e7a2e84d5177194aeb7cadd9f45e421b52bd63a98d26e7a1aa70f3fff5e"}
Nov 04 01:24:42.070807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.070783    4434 scope.go:122] "RemoveContainer" containerID="b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac"
Nov 04 01:24:42.071627 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:24:42.071597    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac\": container with ID starting with b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac not found: ID does not exist" containerID="b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac"
Nov 04 01:24:42.071747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.071637    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac"} err="failed to get container status \"b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac\": rpc error: code = NotFound desc = could not find container \"b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac\": container with ID starting with b0411d932e9386eac051ae89e825e7eeb98bc002b05a6ca4dbbf9d72a34bd7ac not found: ID does not exist"
Nov 04 01:24:42.071747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.071689    4434 scope.go:122] "RemoveContainer" containerID="1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab"
Nov 04 01:24:42.081456 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.081438    4434 scope.go:122] "RemoveContainer" containerID="1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab"
Nov 04 01:24:42.081751 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:24:42.081726    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab\": container with ID starting with 1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab not found: ID does not exist" containerID="1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab"
Nov 04 01:24:42.081838 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.081760    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab"} err="failed to get container status \"1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab\": rpc error: code = NotFound desc = could not find container \"1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab\": container with ID starting with 1668c549c389b793fb0410a1874d858b55abdf56f6c920f5534053007d5620ab not found: ID does not exist"
Nov 04 01:24:42.243209 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.243097    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"]
Nov 04 01:24:42.247288 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.247264    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-b45d1fe0-97c84d5d7-v4mdv"]
Nov 04 01:24:42.363294 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.363257    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"]
Nov 04 01:24:42.369698 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:42.369657    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-b45d1fe0-8c648f698-l297f"]
Nov 04 01:24:43.108889 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:43.108827    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:43.475604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:43.475572    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14420692-cad5-4afe-b05a-ac82e05af453" path="/var/lib/kubelet/pods/14420692-cad5-4afe-b05a-ac82e05af453/volumes"
Nov 04 01:24:43.476095 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:43.476075    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f92b7329-d2bd-4aec-bda1-93be73515f28" path="/var/lib/kubelet/pods/f92b7329-d2bd-4aec-bda1-93be73515f28/volumes"
Nov 04 01:24:47.992215 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:47.992161    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:51.921174 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:24:51.921129    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:24:53.108700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:53.108421    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:24:58.117543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:24:58.117506    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:02.457288 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:02.457228    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:03.556023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.555980    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"]
Nov 04 01:25:03.556500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.556261    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" podUID="a2d6f858-1c1d-4eac-a467-df098059011e" containerName="maas-api" containerID="cri-o://86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a" gracePeriod=30
Nov 04 01:25:03.712502 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.712469    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:03.921948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.921912    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:25:03.964593 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.964556    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca\") pod \"a2d6f858-1c1d-4eac-a467-df098059011e\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") "
Nov 04 01:25:03.964854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.964831    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs\") pod \"a2d6f858-1c1d-4eac-a467-df098059011e\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") "
Nov 04 01:25:03.964997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.964979    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5\" (UniqueName: \"kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5\") pod \"a2d6f858-1c1d-4eac-a467-df098059011e\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") "
Nov 04 01:25:03.965137 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.965121    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\") pod \"a2d6f858-1c1d-4eac-a467-df098059011e\" (UID: \"a2d6f858-1c1d-4eac-a467-df098059011e\") "
Nov 04 01:25:03.965373 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.965061    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca" pod "a2d6f858-1c1d-4eac-a467-df098059011e" (UID: "a2d6f858-1c1d-4eac-a467-df098059011e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:25:03.966340 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.966314    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d6f858-1c1d-4eac-a467-df098059011e-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:25:03.968648 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.968602    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5" pod "a2d6f858-1c1d-4eac-a467-df098059011e" (UID: "a2d6f858-1c1d-4eac-a467-df098059011e"). InnerVolumeSpecName "kube-api-access-2spz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:25:03.969436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.969215    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs" pod "a2d6f858-1c1d-4eac-a467-df098059011e" (UID: "a2d6f858-1c1d-4eac-a467-df098059011e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:25:03.970859 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:03.970825    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls" pod "a2d6f858-1c1d-4eac-a467-df098059011e" (UID: "a2d6f858-1c1d-4eac-a467-df098059011e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:25:04.078717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.068033    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:25:04.078717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.068071    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2spz5\" (UniqueName: \"kubernetes.io/projected/a2d6f858-1c1d-4eac-a467-df098059011e-kube-api-access-2spz5\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:25:04.078717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.068098    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d6f858-1c1d-4eac-a467-df098059011e-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:25:04.148908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.148872    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:04.226639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.226597    4434 generic.go:356] "Generic (PLEG): container finished" podID="a2d6f858-1c1d-4eac-a467-df098059011e" containerID="86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a" exitCode=0
Nov 04 01:25:04.226856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.226646    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"
Nov 04 01:25:04.226856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.226679    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" event={"ID":"a2d6f858-1c1d-4eac-a467-df098059011e","Type":"ContainerDied","Data":"86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a"}
Nov 04 01:25:04.226856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.226747    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq" event={"ID":"a2d6f858-1c1d-4eac-a467-df098059011e","Type":"ContainerDied","Data":"c7832b7af5b92eab67d1e2bb9f05dbbd6bb26712b410b520801ce3cb7553438a"}
Nov 04 01:25:04.226856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.226783    4434 scope.go:122] "RemoveContainer" containerID="86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a"
Nov 04 01:25:04.241376 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.241348    4434 scope.go:122] "RemoveContainer" containerID="86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a"
Nov 04 01:25:04.255217 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:25:04.241799    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a\": container with ID starting with 86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a not found: ID does not exist" containerID="86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a"
Nov 04 01:25:04.255217 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.241839    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a"} err="failed to get container status \"86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a\": rpc error: code = NotFound desc = could not find container \"86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a\": container with ID starting with 86e4c2bec28f1ca0687476227ff0203ce1f0edc0d7735396fc00892cc87fc20a not found: ID does not exist"
Nov 04 01:25:04.406898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.406813    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"]
Nov 04 01:25:04.411286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.411250    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e400a3a6-5b887b8bf5-4mjgq"]
Nov 04 01:25:04.458842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.458812    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:04.842364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:04.842259    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:05.052462 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:05.052419    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:05.276691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:05.272938    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:05.479333 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:05.479297    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2d6f858-1c1d-4eac-a467-df098059011e" path="/var/lib/kubelet/pods/a2d6f858-1c1d-4eac-a467-df098059011e/volumes"
Nov 04 01:25:05.543965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:05.543878    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:06.015719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:06.015245    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:06.374255 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:06.374214    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:06.654573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:06.654484    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:07.061938 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:07.061852    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:07.297399 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:07.297364    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:07.478949 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:07.478913    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:07.821185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:07.821085    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:08.431903 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:08.431857    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:08.765989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:08.765890    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:09.107747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:09.097209    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:09.601169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:09.601135    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:09.797900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:09.797811    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:13.291049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:13.291011    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:13.570985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:13.570878    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:13.935658 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:13.935627    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:14.195755 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:14.195653    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:14.436563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:14.436526    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:16.592579 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.592541    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:16.770711 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.770678    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:16.990215 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990177    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"]
Nov 04 01:25:16.990920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990900    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a2d6f858-1c1d-4eac-a467-df098059011e" containerName="maas-api"
Nov 04 01:25:16.991004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990922    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2d6f858-1c1d-4eac-a467-df098059011e" containerName="maas-api"
Nov 04 01:25:16.991004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990947    4434 cpu_manager.go:414] "containerMap: removing container" podUID="f92b7329-d2bd-4aec-bda1-93be73515f28" containerName="payload-pre-processing"
Nov 04 01:25:16.991004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990955    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f92b7329-d2bd-4aec-bda1-93be73515f28" containerName="payload-pre-processing"
Nov 04 01:25:16.991004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990979    4434 cpu_manager.go:414] "containerMap: removing container" podUID="14420692-cad5-4afe-b05a-ac82e05af453" containerName="payload-processing"
Nov 04 01:25:16.991004 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.990987    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14420692-cad5-4afe-b05a-ac82e05af453" containerName="payload-processing"
Nov 04 01:25:16.991185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.991094    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2d6f858-1c1d-4eac-a467-df098059011e" containerName="maas-api"
Nov 04 01:25:16.991185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.991107    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="f92b7329-d2bd-4aec-bda1-93be73515f28" containerName="payload-pre-processing"
Nov 04 01:25:16.991185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.991118    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="14420692-cad5-4afe-b05a-ac82e05af453" containerName="payload-processing"
Nov 04 01:25:16.996530 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:16.996502    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.058086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.058046    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"]
Nov 04 01:25:17.114734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.113426    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-f10575b3-dockercfg-4lxlk\""
Nov 04 01:25:17.114734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.113581    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-f10575b3\""
Nov 04 01:25:17.116053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.116021    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.116220 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.116164    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l2rd\" (UniqueName: \"kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd\") pod \"payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.217235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.217198    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.217437 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.217326    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l2rd\" (UniqueName: \"kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd\") pod \"payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.218028 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.218006    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.275829 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.275753    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l2rd\" (UniqueName: \"kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd\") pod \"payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.313033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.313000    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:17.721303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:17.721265    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"]
Nov 04 01:25:17.725569 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:25:17.725534    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14b7260_edaa_4fa8_a103_5fa1ec753886.slice/crio-8bfe8a898f7e23ff3a9b3623860b3e37dcb81c15e19f6e11e62a0a7df842d595 WatchSource:0}: Error finding container 8bfe8a898f7e23ff3a9b3623860b3e37dcb81c15e19f6e11e62a0a7df842d595: Status 404 returned error can't find the container with id 8bfe8a898f7e23ff3a9b3623860b3e37dcb81c15e19f6e11e62a0a7df842d595
Nov 04 01:25:18.353588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:18.353554    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9" event={"ID":"d14b7260-edaa-4fa8-a103-5fa1ec753886","Type":"ContainerStarted","Data":"c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d"}
Nov 04 01:25:18.353850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:18.353605    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9" event={"ID":"d14b7260-edaa-4fa8-a103-5fa1ec753886","Type":"ContainerStarted","Data":"8bfe8a898f7e23ff3a9b3623860b3e37dcb81c15e19f6e11e62a0a7df842d595"}
Nov 04 01:25:18.353850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:18.353697    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:18.470639 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:18.470578    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9" podStartSLOduration=2.470563951 podStartE2EDuration="2.470563951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:25:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:25:18.4685687 +0000 UTC m=+1687.130708773" watchObservedRunningTime="2026-11-04 01:25:18.470563951 +0000 UTC m=+1687.132704016"
Nov 04 01:25:18.901106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:18.901045    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:21.671699 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:21.671620    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:21.908686 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:25:21.908628    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:25:21.992864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:21.992772    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:23.993856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:23.993798    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:24.342212 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.342136    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"]
Nov 04 01:25:24.346502 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.346478    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.409591 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.409563    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 04 01:25:24.409753 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.409609    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 04 01:25:24.443239 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443204    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.443394 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443354    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkx8f\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.443444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443396    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.443444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443434    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.443617 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443590    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.443859 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443840    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.443960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.443942    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.455974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.455939    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 04 01:25:24.456153 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.455970    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-7bjdx\""
Nov 04 01:25:24.461962 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.461934    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"]
Nov 04 01:25:24.545342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545312    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.545489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545375    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkx8f\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.545489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545406    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.545489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545433    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.545489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545469    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.545660 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545639    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.545768 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.545734    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.546216 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.546191    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.547790 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.547762    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.547929 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.547906    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.548000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.547965    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.548000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.547970    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.603312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.603250    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.603654 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.603633    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkx8f\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p9rpz\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:24.659131 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:24.659107    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:25.104063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:25.104036    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"]
Nov 04 01:25:25.105463 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:25:25.105432    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981ba35c_3a27_4c04_bce1_490e5c6d1f0a.slice/crio-7c4d755bdf97e37ed3fb7f84dd2d12d80f7d338ffad37cc7271d4b3a700b4b60 WatchSource:0}: Error finding container 7c4d755bdf97e37ed3fb7f84dd2d12d80f7d338ffad37cc7271d4b3a700b4b60: Status 404 returned error can't find the container with id 7c4d755bdf97e37ed3fb7f84dd2d12d80f7d338ffad37cc7271d4b3a700b4b60
Nov 04 01:25:25.438358 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:25.438326    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" event={"ID":"981ba35c-3a27-4c04-bce1-490e5c6d1f0a","Type":"ContainerStarted","Data":"7c4d755bdf97e37ed3fb7f84dd2d12d80f7d338ffad37cc7271d4b3a700b4b60"}
Nov 04 01:25:29.123532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:29.123469    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:29.365748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:29.364450    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:25:31.246735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:31.246703    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:25:31.247129 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:31.246774    4434 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 04 01:25:31.542370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:31.542275    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" event={"ID":"981ba35c-3a27-4c04-bce1-490e5c6d1f0a","Type":"ContainerStarted","Data":"13d7b8bc9a32ff7376c4cc52bce5681cf9f21e39acfe04a97fba0ca94f7fca59"}
Nov 04 01:25:31.542549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:31.542485    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:31.763359 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:31.763310    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" podStartSLOduration=1.625509862 podStartE2EDuration="7.763295772s" totalImagesPullingTime="6.13778591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:25:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:25:31.713011157 +0000 UTC m=+1700.375151210" watchObservedRunningTime="2026-11-04 01:25:31.763295772 +0000 UTC m=+1700.425435832"
Nov 04 01:25:32.489860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:32.489821    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:32.551105 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:32.551078    4434 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-p9rpz container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:25:32.551265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:32.551141    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" podUID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:25:34.425433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:34.425400    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:35.550433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:35.550401    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:25:38.633997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:38.633952    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:39.510833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:39.510788    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:44.479555 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:44.479519    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:49.626940 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:49.626879    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:50.683556 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:50.683527    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:51.715183 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:51.715151    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:51.913469 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:25:51.913433    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:25:51.929198 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:51.929158    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:52.739826 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:52.739792    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:54.684833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:54.684795    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:54.904835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:54.904802    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:55.152997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:55.152953    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:55.457764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:55.457652    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:55.662125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:55.662097    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:55.855393 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:55.855306    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:58.786631 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:58.786594    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:25:58.981285 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:25:58.981050    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:00.422970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:00.422940    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:00.628361 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:00.628328    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:00.799735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:00.799648    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:01.664191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:01.664154    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:01.864363 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:01.864327    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:02.089977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:02.089886    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:05.015796 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:05.015758    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:05.197377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:05.197342    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:05.733941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:05.733884    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:06.104644 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:06.104551    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:06.300374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:06.300338    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:09.805844 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:09.805764    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:09.995915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:09.995876    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:10.187915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:10.187876    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:11.065606 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:11.065572    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:11.245652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:11.245621    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:11.452251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:11.452218    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:17.340539 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:17.340500    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:18.743282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:18.743241    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:18.953135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:18.953093    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:19.127293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.127261    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"]
Nov 04 01:26:19.133237 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.133215    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.188713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.186651    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-5186fe-dockercfg-kpc8m\""
Nov 04 01:26:19.188713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.187831    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-5186fe\""
Nov 04 01:26:19.189176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.189156    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"]
Nov 04 01:26:19.261850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.261811    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc58p\" (UniqueName: \"kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p\") pod \"payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.262049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.261878    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.365817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.365781    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc58p\" (UniqueName: \"kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p\") pod \"payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.366090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.366068    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.367082 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.367054    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.425762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.425695    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc58p\" (UniqueName: \"kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p\") pod \"payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.457675 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.457623    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:19.923179 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:26:19.923147    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24ff5a75_4172_4bf1_8826_97ff4ec4eb2a.slice/crio-5cccb59a8e8b5d106b160c39c894faf477096643200a4ea5000bf918234a5b01 WatchSource:0}: Error finding container 5cccb59a8e8b5d106b160c39c894faf477096643200a4ea5000bf918234a5b01: Status 404 returned error can't find the container with id 5cccb59a8e8b5d106b160c39c894faf477096643200a4ea5000bf918234a5b01
Nov 04 01:26:19.923837 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:19.923811    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"]
Nov 04 01:26:20.904733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:20.904693    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd" event={"ID":"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a","Type":"ContainerStarted","Data":"035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e"}
Nov 04 01:26:20.904954 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:20.904778    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:20.904954 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:20.904795    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd" event={"ID":"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a","Type":"ContainerStarted","Data":"5cccb59a8e8b5d106b160c39c894faf477096643200a4ea5000bf918234a5b01"}
Nov 04 01:26:21.037677 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:21.037611    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd" podStartSLOduration=2.037597348 podStartE2EDuration="2.037597348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:26:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:26:21.032011346 +0000 UTC m=+1749.694151407" watchObservedRunningTime="2026-11-04 01:26:21.037597348 +0000 UTC m=+1749.699737408"
Nov 04 01:26:21.900804 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:26:21.900763    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:26:22.498741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:22.498710    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:27.617383 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:27.617166    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:31.914570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:31.914538    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:26:32.656985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:32.656895    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:37.585604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:37.585571    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:42.460383 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:42.460349    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:47.056315 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.056279    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"]
Nov 04 01:26:47.060647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.060629    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.138762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.138721    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.139000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.138966    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc5t9\" (UniqueName: \"kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.139124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.139100    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.139177 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.139159    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.164735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.164700    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-9a0432-serving-cert\""
Nov 04 01:26:47.168696 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.168650    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"]
Nov 04 01:26:47.240973 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.240943    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.241141 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.241050    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc5t9\" (UniqueName: \"kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.241141 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.241090    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.241141 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:26:47.241092    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-serving-cert: secret "maas-api-e2e-shared-a-w6-9a0432-serving-cert" not found
Nov 04 01:26:47.241141 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.241117    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.241359 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:26:47.241166    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls podName:b9e972b7-1b70-4122-a7d5-c45e327d6281 nodeName:}" failed. No retries permitted until 2026-11-04 01:26:47.741147169 +0000 UTC m=+1776.403287223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls") pod "maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281") : secret "maas-api-e2e-shared-a-w6-9a0432-serving-cert" not found
Nov 04 01:26:47.241801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.241779    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.243470 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.243450    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.299962 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.299934    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc5t9\" (UniqueName: \"kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.307913 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.307829    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:47.506938 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.506878    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:47.686344 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.686306    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:47.750863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.750832    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.753255 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.753227    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:47.974722 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:47.974629    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:48.355156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:48.355128    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"]
Nov 04 01:26:48.357814 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:26:48.357786    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e972b7_1b70_4122_a7d5_c45e327d6281.slice/crio-5c2d608ae693470df3fcf74a7cd3fbe1ad0e1cb2c67b48fd5f67d6634a25bf11 WatchSource:0}: Error finding container 5c2d608ae693470df3fcf74a7cd3fbe1ad0e1cb2c67b48fd5f67d6634a25bf11: Status 404 returned error can't find the container with id 5c2d608ae693470df3fcf74a7cd3fbe1ad0e1cb2c67b48fd5f67d6634a25bf11
Nov 04 01:26:49.092733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:49.092695    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" event={"ID":"b9e972b7-1b70-4122-a7d5-c45e327d6281","Type":"ContainerStarted","Data":"5c2d608ae693470df3fcf74a7cd3fbe1ad0e1cb2c67b48fd5f67d6634a25bf11"}
Nov 04 01:26:50.101268 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:50.101226    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" event={"ID":"b9e972b7-1b70-4122-a7d5-c45e327d6281","Type":"ContainerStarted","Data":"9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344"}
Nov 04 01:26:50.101729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:50.101304    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:50.219822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:50.219766    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" podStartSLOduration=2.307582562 podStartE2EDuration="3.219746995s" totalImagesPullingTime="912.164433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:26:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:26:50.217595557 +0000 UTC m=+1778.879735619" watchObservedRunningTime="2026-11-04 01:26:50.219746995 +0000 UTC m=+1778.881887056"
Nov 04 01:26:51.900182 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:26:51.900138    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:26:52.428944 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:52.428899    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:56.116293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:56.116260    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:26:57.358443 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:57.358395    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:57.547187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:57.547156    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:26:57.741650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:26:57.741616    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:02.755297 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:02.755266    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:03.128143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:03.128103    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:05.476512 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:05.476481    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:07.653480 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:07.653444    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"]
Nov 04 01:27:07.702974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:07.702940    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"]
Nov 04 01:27:07.761396 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:07.761358    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:09.023033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:09.022994    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:13.609214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:13.609174    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"]
Nov 04 01:27:13.902006 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:13.901970    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:14.072295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.072258    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:14.190219 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.190141    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"]
Nov 04 01:27:14.194856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.194830    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.247762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.247723    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-5186fe-serving-cert\""
Nov 04 01:27:14.249719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.249690    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"]
Nov 04 01:27:14.295633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.295602    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.295803 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.295761    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.295847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.295826    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.295925 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.295909    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49rkd\" (UniqueName: \"kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.397301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.397268    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.397494 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.397371    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.397494 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.397427    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.397494 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.397470    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49rkd\" (UniqueName: \"kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.397717 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:14.397532    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-serving-cert: secret "maas-api-e2e-shared-a-w0-5186fe-serving-cert" not found
Nov 04 01:27:14.397717 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:14.397604    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls podName:623be538-08d1-4ad5-806c-c7b5383d9a32 nodeName:}" failed. No retries permitted until 2026-11-04 01:27:14.897588917 +0000 UTC m=+1803.559728970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls") pod "maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" (UID: "623be538-08d1-4ad5-806c-c7b5383d9a32") : secret "maas-api-e2e-shared-a-w0-5186fe-serving-cert" not found
Nov 04 01:27:14.398101 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.398081    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.399759 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.399744    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.431695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.431654    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:14.456297 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.456247    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49rkd\" (UniqueName: \"kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.644691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.639026    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:14.818717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.818620    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:14.906882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.906853    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:14.909351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:14.909327    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:15.120518 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:15.120337    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:15.624385 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:15.624358    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"]
Nov 04 01:27:15.626931 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:27:15.626890    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623be538_08d1_4ad5_806c_c7b5383d9a32.slice/crio-dc14ee58bcc37efa7f16869ecee053d68a5bf925c3ec748df5393dcf8cfe60c1 WatchSource:0}: Error finding container dc14ee58bcc37efa7f16869ecee053d68a5bf925c3ec748df5393dcf8cfe60c1: Status 404 returned error can't find the container with id dc14ee58bcc37efa7f16869ecee053d68a5bf925c3ec748df5393dcf8cfe60c1
Nov 04 01:27:16.314655 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:16.314620    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" event={"ID":"623be538-08d1-4ad5-806c-c7b5383d9a32","Type":"ContainerStarted","Data":"dc14ee58bcc37efa7f16869ecee053d68a5bf925c3ec748df5393dcf8cfe60c1"}
Nov 04 01:27:17.322331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:17.322280    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" event={"ID":"623be538-08d1-4ad5-806c-c7b5383d9a32","Type":"ContainerStarted","Data":"9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74"}
Nov 04 01:27:17.322794 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:17.322533    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:17.442759 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:17.442700    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" podStartSLOduration=2.513258472 podStartE2EDuration="3.4426847s" totalImagesPullingTime="929.426228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:27:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:27:17.440699537 +0000 UTC m=+1806.102839598" watchObservedRunningTime="2026-11-04 01:27:17.4426847 +0000 UTC m=+1806.104824760"
Nov 04 01:27:18.610314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.610246    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9" podUID="d14b7260-edaa-4fa8-a103-5fa1ec753886" containerName="payload-pre-processing" containerID="cri-o://c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d" gracePeriod=25
Nov 04 01:27:18.902263 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.902230    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:27:18.990212 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.990185    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume\") pod \"d14b7260-edaa-4fa8-a103-5fa1ec753886\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") "
Nov 04 01:27:18.990395 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.990245    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd\" (UniqueName: \"kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd\") pod \"d14b7260-edaa-4fa8-a103-5fa1ec753886\" (UID: \"d14b7260-edaa-4fa8-a103-5fa1ec753886\") "
Nov 04 01:27:18.990598 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.990564    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume" pod "d14b7260-edaa-4fa8-a103-5fa1ec753886" (UID: "d14b7260-edaa-4fa8-a103-5fa1ec753886"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:18.990795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.990781    4434 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d14b7260-edaa-4fa8-a103-5fa1ec753886-plugins-config-volume\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:18.992400 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:18.992380    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd" pod "d14b7260-edaa-4fa8-a103-5fa1ec753886" (UID: "d14b7260-edaa-4fa8-a103-5fa1ec753886"). InnerVolumeSpecName "kube-api-access-5l2rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:19.092210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.092179    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l2rd\" (UniqueName: \"kubernetes.io/projected/d14b7260-edaa-4fa8-a103-5fa1ec753886-kube-api-access-5l2rd\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:19.335911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.335873    4434 generic.go:356] "Generic (PLEG): container finished" podID="d14b7260-edaa-4fa8-a103-5fa1ec753886" containerID="c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d" exitCode=0
Nov 04 01:27:19.335911 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.335914    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"
Nov 04 01:27:19.336144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.335945    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9" event={"ID":"d14b7260-edaa-4fa8-a103-5fa1ec753886","Type":"ContainerDied","Data":"c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d"}
Nov 04 01:27:19.336144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.336009    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9" event={"ID":"d14b7260-edaa-4fa8-a103-5fa1ec753886","Type":"ContainerDied","Data":"8bfe8a898f7e23ff3a9b3623860b3e37dcb81c15e19f6e11e62a0a7df842d595"}
Nov 04 01:27:19.336144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.336032    4434 scope.go:122] "RemoveContainer" containerID="c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d"
Nov 04 01:27:19.346142 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.346114    4434 scope.go:122] "RemoveContainer" containerID="c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d"
Nov 04 01:27:19.346397 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:19.346375    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d\": container with ID starting with c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d not found: ID does not exist" containerID="c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d"
Nov 04 01:27:19.346453 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.346405    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d"} err="failed to get container status \"c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d\": rpc error: code = NotFound desc = could not find container \"c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d\": container with ID starting with c5b4903564d8f2be77507f49ab30e55a3466f2fcbaeced0567175efabd16390d not found: ID does not exist"
Nov 04 01:27:19.504721 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.504676    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"]
Nov 04 01:27:19.510466 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:19.510442    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-f10575b3-5bb4ffc567-4cjw9"]
Nov 04 01:27:20.105096 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:20.105066    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:21.475760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:21.475729    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d14b7260-edaa-4fa8-a103-5fa1ec753886" path="/var/lib/kubelet/pods/d14b7260-edaa-4fa8-a103-5fa1ec753886/volumes"
Nov 04 01:27:21.898286 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:21.898253    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:27:22.676082 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:22.676011    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main" containerID="cri-o://fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a" gracePeriod=45
Nov 04 01:27:22.724488 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:22.724445    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" containerID="cri-o://f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7" gracePeriod=45
Nov 04 01:27:22.735074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:22.735043    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": dial tcp 10.134.0.55:8000: connect: connection refused"
Nov 04 01:27:23.056336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.056307    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:27:23.059824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.059794    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:27:23.155236 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155202    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs\") pod \"9fff6230-5d60-49e0-b116-3f060d15a052\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") "
Nov 04 01:27:23.155404 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155307    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home\") pod \"4a3ab8da-14e4-4a0f-8300-88850af83969\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") "
Nov 04 01:27:23.155404 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155331    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm\") pod \"9fff6230-5d60-49e0-b116-3f060d15a052\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") "
Nov 04 01:27:23.155404 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155396    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c\" (UniqueName: \"kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c\") pod \"4a3ab8da-14e4-4a0f-8300-88850af83969\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") "
Nov 04 01:27:23.155530 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155473    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm\") pod \"4a3ab8da-14e4-4a0f-8300-88850af83969\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") "
Nov 04 01:27:23.155530 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155511    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir\") pod \"4a3ab8da-14e4-4a0f-8300-88850af83969\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") "
Nov 04 01:27:23.155608 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155577    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir\") pod \"9fff6230-5d60-49e0-b116-3f060d15a052\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") "
Nov 04 01:27:23.155681 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155604    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home" pod "4a3ab8da-14e4-4a0f-8300-88850af83969" (UID: "4a3ab8da-14e4-4a0f-8300-88850af83969"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.155681 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155619    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg\" (UniqueName: \"kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg\") pod \"9fff6230-5d60-49e0-b116-3f060d15a052\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") "
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155880    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir" pod "9fff6230-5d60-49e0-b116-3f060d15a052" (UID: "9fff6230-5d60-49e0-b116-3f060d15a052"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.155886    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir" pod "4a3ab8da-14e4-4a0f-8300-88850af83969" (UID: "4a3ab8da-14e4-4a0f-8300-88850af83969"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156026    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache\") pod \"4a3ab8da-14e4-4a0f-8300-88850af83969\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") "
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156065    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs\") pod \"4a3ab8da-14e4-4a0f-8300-88850af83969\" (UID: \"4a3ab8da-14e4-4a0f-8300-88850af83969\") "
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156099    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache\") pod \"9fff6230-5d60-49e0-b116-3f060d15a052\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") "
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156160    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home\") pod \"9fff6230-5d60-49e0-b116-3f060d15a052\" (UID: \"9fff6230-5d60-49e0-b116-3f060d15a052\") "
Nov 04 01:27:23.156330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156322    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache" pod "4a3ab8da-14e4-4a0f-8300-88850af83969" (UID: "4a3ab8da-14e4-4a0f-8300-88850af83969"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.156959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156639    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache" pod "9fff6230-5d60-49e0-b116-3f060d15a052" (UID: "9fff6230-5d60-49e0-b116-3f060d15a052"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.156959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.156895    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home" pod "9fff6230-5d60-49e0-b116-3f060d15a052" (UID: "9fff6230-5d60-49e0-b116-3f060d15a052"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.157102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.157049    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.157102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.157069    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.157102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.157084    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.157102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.157099    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.157323 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.157112    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.157323 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.157126    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.158303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.158268    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm" pod "4a3ab8da-14e4-4a0f-8300-88850af83969" (UID: "4a3ab8da-14e4-4a0f-8300-88850af83969"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.158303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.158276    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c" pod "4a3ab8da-14e4-4a0f-8300-88850af83969" (UID: "4a3ab8da-14e4-4a0f-8300-88850af83969"). InnerVolumeSpecName "kube-api-access-fff6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:23.158543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.158526    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs" pod "4a3ab8da-14e4-4a0f-8300-88850af83969" (UID: "4a3ab8da-14e4-4a0f-8300-88850af83969"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:23.158543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.158535    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm" pod "9fff6230-5d60-49e0-b116-3f060d15a052" (UID: "9fff6230-5d60-49e0-b116-3f060d15a052"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:27:23.158764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.158750    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs" pod "9fff6230-5d60-49e0-b116-3f060d15a052" (UID: "9fff6230-5d60-49e0-b116-3f060d15a052"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:27:23.159220 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.159200    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg" pod "9fff6230-5d60-49e0-b116-3f060d15a052" (UID: "9fff6230-5d60-49e0-b116-3f060d15a052"). InnerVolumeSpecName "kube-api-access-kvtlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:23.259184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.259103    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvtlg\" (UniqueName: \"kubernetes.io/projected/9fff6230-5d60-49e0-b116-3f060d15a052-kube-api-access-kvtlg\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.259184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.259142    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a3ab8da-14e4-4a0f-8300-88850af83969-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.259184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.259158    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9fff6230-5d60-49e0-b116-3f060d15a052-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.259184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.259176    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9fff6230-5d60-49e0-b116-3f060d15a052-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.259421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.259191    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fff6c\" (UniqueName: \"kubernetes.io/projected/4a3ab8da-14e4-4a0f-8300-88850af83969-kube-api-access-fff6c\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.259421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.259205    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a3ab8da-14e4-4a0f-8300-88850af83969-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:23.335659 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.335632    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:27:23.364091 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.364060    4434 generic.go:356] "Generic (PLEG): container finished" podID="9fff6230-5d60-49e0-b116-3f060d15a052" containerID="f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7" exitCode=0
Nov 04 01:27:23.364251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.364099    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"
Nov 04 01:27:23.364251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.364136    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" event={"ID":"9fff6230-5d60-49e0-b116-3f060d15a052","Type":"ContainerDied","Data":"f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7"}
Nov 04 01:27:23.364251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.364183    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z" event={"ID":"9fff6230-5d60-49e0-b116-3f060d15a052","Type":"ContainerDied","Data":"501253abbccf67aef1c1a350640c8eae760d797e9172ee3994bd1c079f184af0"}
Nov 04 01:27:23.364251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.364207    4434 scope.go:122] "RemoveContainer" containerID="f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7"
Nov 04 01:27:23.365517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.365493    4434 generic.go:356] "Generic (PLEG): container finished" podID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerID="fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a" exitCode=0
Nov 04 01:27:23.365637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.365518    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" event={"ID":"4a3ab8da-14e4-4a0f-8300-88850af83969","Type":"ContainerDied","Data":"fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a"}
Nov 04 01:27:23.365637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.365544    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"
Nov 04 01:27:23.365637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.365556    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9" event={"ID":"4a3ab8da-14e4-4a0f-8300-88850af83969","Type":"ContainerDied","Data":"a3fb4b06bba6ea4c52ea1c5f12056e2ad88bbd8b6a9a8fa3b366e3f47ab8f039"}
Nov 04 01:27:23.375464 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.375445    4434 scope.go:122] "RemoveContainer" containerID="f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7"
Nov 04 01:27:23.376059 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:23.375776    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7\": container with ID starting with f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7 not found: ID does not exist" containerID="f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7"
Nov 04 01:27:23.376151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.376075    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7"} err="failed to get container status \"f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7\": rpc error: code = NotFound desc = could not find container \"f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7\": container with ID starting with f257518c37548d592f543b6bb154197f14015e1c708398f43970f86397d045c7 not found: ID does not exist"
Nov 04 01:27:23.376151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.376099    4434 scope.go:122] "RemoveContainer" containerID="fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a"
Nov 04 01:27:23.384633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.384615    4434 scope.go:122] "RemoveContainer" containerID="fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a"
Nov 04 01:27:23.384891 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:23.384875    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a\": container with ID starting with fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a not found: ID does not exist" containerID="fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a"
Nov 04 01:27:23.384944 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.384898    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a"} err="failed to get container status \"fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a\": rpc error: code = NotFound desc = could not find container \"fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a\": container with ID starting with fde6b7c16d49f2a0a47323f0e67036055c30be779fd7acfe891dab9f2df6b07a not found: ID does not exist"
Nov 04 01:27:23.580978 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.580874    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"]
Nov 04 01:27:23.584417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.584390    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-w3-2ee045-kserve-77db54895c-gj87z"]
Nov 04 01:27:23.684220 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.684189    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"]
Nov 04 01:27:23.687449 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:23.687425    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-2ee045/distinct-2-w3-2ee045-kserve-66866f894-l4hh9"]
Nov 04 01:27:24.911412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:24.911368    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"]
Nov 04 01:27:25.206754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:25.206644    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:25.386457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:25.386424    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:25.476799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:25.476719    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" path="/var/lib/kubelet/pods/4a3ab8da-14e4-4a0f-8300-88850af83969/volumes"
Nov 04 01:27:25.477277 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:25.477255    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" path="/var/lib/kubelet/pods/9fff6230-5d60-49e0-b116-3f060d15a052/volumes"
Nov 04 01:27:25.814370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:25.814279    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:27.945623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:27.945593    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:29.911932 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:29.911870    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerName="payload-pre-processing" containerID="cri-o://f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef" gracePeriod=25
Nov 04 01:27:30.748845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:30.748809    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:30.942870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:30.942833    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:30.968264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:30.968222    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:27:31.228316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:31.228282    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:32.559054 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:32.559019    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:32.735593 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:32.735556    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:32.965178 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:32.965135    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:34.189364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:34.189324    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:36.275170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:36.275137    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:37.695628 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:37.695597    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:39.692786 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:39.692745    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:39.887984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:39.887949    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:40.967932 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:40.967892    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 04 01:27:41.438063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:41.438028    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:46.710841 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:46.710801    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:50.185331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.185267    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:27:50.301064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.301029    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s\" (UniqueName: \"kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s\") pod \"2fc5756b-93da-49c1-af2b-e0f5e341609d\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") "
Nov 04 01:27:50.301303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.301273    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume\") pod \"2fc5756b-93da-49c1-af2b-e0f5e341609d\" (UID: \"2fc5756b-93da-49c1-af2b-e0f5e341609d\") "
Nov 04 01:27:50.302142 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.302020    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume" pod "2fc5756b-93da-49c1-af2b-e0f5e341609d" (UID: "2fc5756b-93da-49c1-af2b-e0f5e341609d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:27:50.304999 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.304974    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s" pod "2fc5756b-93da-49c1-af2b-e0f5e341609d" (UID: "2fc5756b-93da-49c1-af2b-e0f5e341609d"). InnerVolumeSpecName "kube-api-access-c8n2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:27:50.403531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.403498    4434 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2fc5756b-93da-49c1-af2b-e0f5e341609d-plugins-config-volume\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.403531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.403526    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8n2s\" (UniqueName: \"kubernetes.io/projected/2fc5756b-93da-49c1-af2b-e0f5e341609d-kube-api-access-c8n2s\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:27:50.560147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.560063    4434 generic.go:356] "Generic (PLEG): container finished" podID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerID="f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef" exitCode=0
Nov 04 01:27:50.560147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.560111    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"
Nov 04 01:27:50.560339 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.560146    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" event={"ID":"2fc5756b-93da-49c1-af2b-e0f5e341609d","Type":"ContainerDied","Data":"f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef"}
Nov 04 01:27:50.560339 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.560203    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls" event={"ID":"2fc5756b-93da-49c1-af2b-e0f5e341609d","Type":"ContainerDied","Data":"400fbd22f4f31732f87e46311e2963505a588e2022568390a67a64b1f9f5fb73"}
Nov 04 01:27:50.560339 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.560228    4434 scope.go:122] "RemoveContainer" containerID="f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef"
Nov 04 01:27:50.570050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.570033    4434 scope.go:122] "RemoveContainer" containerID="f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef"
Nov 04 01:27:50.570305 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:50.570290    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef\": container with ID starting with f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef not found: ID does not exist" containerID="f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef"
Nov 04 01:27:50.570348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.570315    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef"} err="failed to get container status \"f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef\": rpc error: code = NotFound desc = could not find container \"f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef\": container with ID starting with f8578a60163a6e065152bf63b98d30f8ee14633bb1aecb1d3df4181e629d6bef not found: ID does not exist"
Nov 04 01:27:50.730992 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.730955    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"]
Nov 04 01:27:50.735177 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:50.735148    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-2ee045-7fd8b7f4dc-tmsls"]
Nov 04 01:27:51.477261 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:51.477227    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" path="/var/lib/kubelet/pods/2fc5756b-93da-49c1-af2b-e0f5e341609d/volumes"
Nov 04 01:27:51.700686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:51.697790    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:51.900536 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:27:51.900490    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:27:56.029044 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:56.028771    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:57.862410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:57.862371    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:58.177965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:58.177928    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:58.426244 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:58.426208    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:58.682688 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:58.682625    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:59.410288 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:59.410250    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:27:59.890496 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:27:59.890455    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:00.211581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:00.211488    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:01.111981 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:01.111928    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:03.270190 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:03.270113    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:04.706823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:04.706793    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:04.934271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:04.934237    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:05.535741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:05.530175    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:05.716955 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:05.716920    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:05.898038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:05.898008    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:06.083450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:06.083420    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:06.264780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:06.264699    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:06.826160 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:06.826133    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:07.381050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:07.381007    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:07.670971 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:07.670874    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:08.366840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:08.366800    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:08.560379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:08.560348    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:09.020755 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:09.017127    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:09.202417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:09.202384    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:09.425603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:09.425571    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:11.404119 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:11.404089    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:11.613627 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:11.613574    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:11.769124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:11.769008    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:13.795528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:13.795476    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:14.033419 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:14.033382    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:14.226559 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:14.226510    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:15.501499 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:15.501464    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:15.669582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:15.669534    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:16.367195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:16.367159    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:16.571512 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:16.571471    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:17.144282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:17.144227    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:21.905573 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:21.905529    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:28:22.717119 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:22.717087    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"]
Nov 04 01:28:22.717312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:22.717293    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy" containerID="cri-o://8a94fb4ea22c838477362ac4cc24cdd78dc950c815e85ccc5fd08c306b37e10b" gracePeriod=30
Nov 04 01:28:22.860921 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:22.860881    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:23.106369 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.106284    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:23.226293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.226261    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"]
Nov 04 01:28:23.226551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.226522    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" containerID="cri-o://7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2" gracePeriod=30
Nov 04 01:28:23.230172 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.230148    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"]
Nov 04 01:28:23.230345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.230324    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy" containerID="cri-o://7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6" gracePeriod=30
Nov 04 01:28:23.277644 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.277572    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"]
Nov 04 01:28:23.277926 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.277886    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" containerID="cri-o://59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5" gracePeriod=30
Nov 04 01:28:23.282890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.282856    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"]
Nov 04 01:28:23.283108 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.283082    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy" containerID="cri-o://429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730" gracePeriod=30
Nov 04 01:28:23.500030 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.499995    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:23.991987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.991951    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:28:23.992157 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:23.992012    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:28:24.509879 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:24.509847    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:24.743552 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:24.743518    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:26.125691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:26.119144    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:26.433266 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:26.433096    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:26.616690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:26.608064    4434 patch_prober.go:29] interesting pod/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:28:26.616690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:26.608130    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:28:27.872449 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:27.872417    4434 generic.go:356] "Generic (PLEG): container finished" podID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerID="8a94fb4ea22c838477362ac4cc24cdd78dc950c815e85ccc5fd08c306b37e10b" exitCode=0
Nov 04 01:28:27.872840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:27.872486    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" event={"ID":"3e582e99-2e83-43a3-b470-5929d857ffe8","Type":"ContainerDied","Data":"8a94fb4ea22c838477362ac4cc24cdd78dc950c815e85ccc5fd08c306b37e10b"}
Nov 04 01:28:27.999141 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:27.999040    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:28:27.999442 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:27.999416    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:28.130371 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.130327    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.130915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.130878    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:28.131048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.130949    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131001    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131137 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131079    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131137 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131110    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131237 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131155    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131237 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131216    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131472 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131388    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.131602 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.131570    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo\") pod \"3e582e99-2e83-43a3-b470-5929d857ffe8\" (UID: \"3e582e99-2e83-43a3-b470-5929d857ffe8\") "
Nov 04 01:28:28.132481 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.132454    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e582e99-2e83-43a3-b470-5929d857ffe8-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.132986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.132963    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.133602 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.133097    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.133778 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.133326    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.133885 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.133365    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.135857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.135834    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.136493 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.136468    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:28.138130 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.138097    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "kube-api-access-vn2pv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.143339 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.142527    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token" pod "3e582e99-2e83-43a3-b470-5929d857ffe8" (UID: "3e582e99-2e83-43a3-b470-5929d857ffe8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.233582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233549    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233578    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233588    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233598    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233607    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233620    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233633    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e582e99-2e83-43a3-b470-5929d857ffe8-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.233830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.233642    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn2pv\" (UniqueName: \"kubernetes.io/projected/3e582e99-2e83-43a3-b470-5929d857ffe8-kube-api-access-vn2pv\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.723960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.723337    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:28:28.861073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861035    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861238 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861092    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861238 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861163    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861238 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861208    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861259    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861415    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861506    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861593    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.861704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.861619    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs\") pod \"c0c47b89-8d4d-425c-aada-e65032ca072a\" (UID: \"c0c47b89-8d4d-425c-aada-e65032ca072a\") "
Nov 04 01:28:28.862973 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.862781    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.862973 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.862898    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:28.863257 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.863159    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.863473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.863369    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.863473 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.863440    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.863875 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.863840    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.866822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.866791    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:28.866984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.866963    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "kube-api-access-lcltv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.867722 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.867699    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy" pod "c0c47b89-8d4d-425c-aada-e65032ca072a" (UID: "c0c47b89-8d4d-425c-aada-e65032ca072a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.881761 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.881732    4434 generic.go:356] "Generic (PLEG): container finished" podID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerID="7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6" exitCode=0
Nov 04 01:28:28.882114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.881798    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"
Nov 04 01:28:28.882114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.881825    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" event={"ID":"c0c47b89-8d4d-425c-aada-e65032ca072a","Type":"ContainerDied","Data":"7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6"}
Nov 04 01:28:28.882114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.882100    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:28:28.882389 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.882365    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5" event={"ID":"c0c47b89-8d4d-425c-aada-e65032ca072a","Type":"ContainerDied","Data":"03b6737b394d367375494fa712d43cada505a96a1480ac44eac6f465dd86a790"}
Nov 04 01:28:28.882472 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.882425    4434 scope.go:122] "RemoveContainer" containerID="7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6"
Nov 04 01:28:28.883709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.883477    4434 generic.go:356] "Generic (PLEG): container finished" podID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerID="59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5" exitCode=0
Nov 04 01:28:28.883709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.883598    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" event={"ID":"8ace3819-81cc-4578-9d47-108614bb3cf7","Type":"ContainerDied","Data":"59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5"}
Nov 04 01:28:28.883709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.883649    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c" event={"ID":"8ace3819-81cc-4578-9d47-108614bb3cf7","Type":"ContainerDied","Data":"04e46864495d006899c1943daeb319ac02a0061a0b7bf2e9aedfea3df01d65b2"}
Nov 04 01:28:28.885083 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.885064    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:28:28.891600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.891011    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l" event={"ID":"3e582e99-2e83-43a3-b470-5929d857ffe8","Type":"ContainerDied","Data":"cca491e75dc9a8506090181cd416258ecea899f39c543bb2f3829a39dd426d34"}
Nov 04 01:28:28.891600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.891116    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"
Nov 04 01:28:28.892797 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.892777    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:28:28.898841 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.897948    4434 scope.go:122] "RemoveContainer" containerID="7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6"
Nov 04 01:28:28.899170 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:28.899056    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6\": container with ID starting with 7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6 not found: ID does not exist" containerID="7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6"
Nov 04 01:28:28.899170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.899090    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6"} err="failed to get container status \"7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6\": rpc error: code = NotFound desc = could not find container \"7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6\": container with ID starting with 7cd38a522fc063a062d3dc4586a933a574334a9fc8491cd59c370f8f988783a6 not found: ID does not exist"
Nov 04 01:28:28.899170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.899114    4434 scope.go:122] "RemoveContainer" containerID="59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5"
Nov 04 01:28:28.901188 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.901165    4434 generic.go:356] "Generic (PLEG): container finished" podID="c16b56ea-40c6-4e0a-a895-67422da97231" containerID="429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730" exitCode=0
Nov 04 01:28:28.902360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.902339    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" event={"ID":"c16b56ea-40c6-4e0a-a895-67422da97231","Type":"ContainerDied","Data":"429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730"}
Nov 04 01:28:28.902729 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.902707    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf" event={"ID":"c16b56ea-40c6-4e0a-a895-67422da97231","Type":"ContainerDied","Data":"e98e73a7e299e03b2261dbb101dffeee48ff39c49f8387d03b3b437603569e6d"}
Nov 04 01:28:28.904576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.904476    4434 generic.go:356] "Generic (PLEG): container finished" podID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerID="7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2" exitCode=0
Nov 04 01:28:28.904576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.904528    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" event={"ID":"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1","Type":"ContainerDied","Data":"7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2"}
Nov 04 01:28:28.904576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.904563    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l" event={"ID":"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1","Type":"ContainerDied","Data":"d118fe19ddb0ef5fb0d9f6c473ecc350a78c76601a971980d2e1cb6a4674525b"}
Nov 04 01:28:28.904862 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.904629    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"
Nov 04 01:28:28.925312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.925282    4434 scope.go:122] "RemoveContainer" containerID="59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5"
Nov 04 01:28:28.926174 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:28.926086    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5\": container with ID starting with 59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5 not found: ID does not exist" containerID="59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5"
Nov 04 01:28:28.926387 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.926355    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5"} err="failed to get container status \"59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5\": rpc error: code = NotFound desc = could not find container \"59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5\": container with ID starting with 59c3a3f4c4d1824a0a46dc66a9b61894a8006e7aeb4a8ff24bb482da49a45cb5 not found: ID does not exist"
Nov 04 01:28:28.926502 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.926489    4434 scope.go:122] "RemoveContainer" containerID="8a94fb4ea22c838477362ac4cc24cdd78dc950c815e85ccc5fd08c306b37e10b"
Nov 04 01:28:28.941500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.941481    4434 scope.go:122] "RemoveContainer" containerID="429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730"
Nov 04 01:28:28.966212 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966176    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.966429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966279    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.966429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966318    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.966429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966349    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.966429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966383    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.966429 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966418    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.966638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966457    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.966638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966500    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.966638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966564    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.966638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966594    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.966638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966623    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.966833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966681    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.966833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966720    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.966833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966756    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.966985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966876    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.966985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966909    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.966985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.966976    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.967097 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967028    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.967097 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967082    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.967179 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967154    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.967229 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967200    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.967229 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967219    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token\") pod \"c16b56ea-40c6-4e0a-a895-67422da97231\" (UID: \"c16b56ea-40c6-4e0a-a895-67422da97231\") "
Nov 04 01:28:28.967325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967253    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.967325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967296    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.967325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967318    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.967461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967376    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket\") pod \"8ace3819-81cc-4578-9d47-108614bb3cf7\" (UID: \"8ace3819-81cc-4578-9d47-108614bb3cf7\") "
Nov 04 01:28:28.967461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967396    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert\") pod \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\" (UID: \"db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1\") "
Nov 04 01:28:28.967941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967925    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.967986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967943    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.967986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967955    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.967986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967964    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.967986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967973    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0c47b89-8d4d-425c-aada-e65032ca072a-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.967986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967983    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.968150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.967992    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.968150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968002    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcltv\" (UniqueName: \"kubernetes.io/projected/c0c47b89-8d4d-425c-aada-e65032ca072a-kube-api-access-lcltv\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.968150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968010    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0c47b89-8d4d-425c-aada-e65032ca072a-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:28.968559 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968529    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.968777 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968702    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.968777 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968719    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:28.968945 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968902    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.968945 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968921    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:28.969056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.968999    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.969210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.969189    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.971543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.971517    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.971811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.971787    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.972029 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972007    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.972121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972085    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.972192 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972141    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:28.972261 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972237    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "kube-api-access-slzsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.972371 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972296    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.972465 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972448    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.972527 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972471    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo" pod "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" (UID: "db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:28.972649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972631    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.972786 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972765    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:28.972880 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972858    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.972943 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.972865    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.973060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.973044    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.974016 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.973991    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:28.974106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.974090    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.974631 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.974597    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "kube-api-access-jqs2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.975071 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.975047    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy" pod "c16b56ea-40c6-4e0a-a895-67422da97231" (UID: "c16b56ea-40c6-4e0a-a895-67422da97231"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:28.975407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.975384    4434 scope.go:122] "RemoveContainer" containerID="429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730"
Nov 04 01:28:28.975871 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:28.975838    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730\": container with ID starting with 429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730 not found: ID does not exist" containerID="429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730"
Nov 04 01:28:28.975970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.975873    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730"} err="failed to get container status \"429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730\": rpc error: code = NotFound desc = could not find container \"429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730\": container with ID starting with 429bf777e05dd02b171252f7f85d486393af24b566da72190b78665c8f5e1730 not found: ID does not exist"
Nov 04 01:28:28.975970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.975898    4434 scope.go:122] "RemoveContainer" containerID="7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2"
Nov 04 01:28:28.976247 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.976225    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "kube-api-access-swhvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.976541 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.976517    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token" pod "8ace3819-81cc-4578-9d47-108614bb3cf7" (UID: "8ace3819-81cc-4578-9d47-108614bb3cf7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:28.986741 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.986722    4434 scope.go:122] "RemoveContainer" containerID="7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2"
Nov 04 01:28:28.987040 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:28.987020    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2\": container with ID starting with 7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2 not found: ID does not exist" containerID="7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2"
Nov 04 01:28:28.987098 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:28.987047    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2"} err="failed to get container status \"7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2\": rpc error: code = NotFound desc = could not find container \"7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2\": container with ID starting with 7c0b6951a8cc8d391b486d9d4f0a24d96dee50566e502c742fc02ccb68c406a2 not found: ID does not exist"
Nov 04 01:28:29.071254 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071213    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071254 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071255    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071270    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqs2x\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-kube-api-access-jqs2x\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071286    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071299    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slzsr\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-kube-api-access-slzsr\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071314    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c16b56ea-40c6-4e0a-a895-67422da97231-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071329    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071342    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071356    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071370    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071385    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071399    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071412    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ace3819-81cc-4578-9d47-108614bb3cf7-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071425    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071439    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071454    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071469    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swhvm\" (UniqueName: \"kubernetes.io/projected/8ace3819-81cc-4578-9d47-108614bb3cf7-kube-api-access-swhvm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071482    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c16b56ea-40c6-4e0a-a895-67422da97231-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071500    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071514    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071526    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c16b56ea-40c6-4e0a-a895-67422da97231-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.071531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071539    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ace3819-81cc-4578-9d47-108614bb3cf7-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.074422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071553    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.074422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071566    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.074422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071579    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.074422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071594    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ace3819-81cc-4578-9d47-108614bb3cf7-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.074422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.071607    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c16b56ea-40c6-4e0a-a895-67422da97231-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:29.514498 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.514465    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"]
Nov 04 01:28:29.518435 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.518405    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-4xjb5"]
Nov 04 01:28:29.620656 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.620619    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"]
Nov 04 01:28:29.622522 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.622492    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:29.625373 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.625350    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-7x55l"]
Nov 04 01:28:29.735317 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.735247    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"]
Nov 04 01:28:29.740500 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.740471    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-m465l"]
Nov 04 01:28:29.901243 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.901204    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:29.912789 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.912763    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"
Nov 04 01:28:29.916008 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:29.915988    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"
Nov 04 01:28:30.077058 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.076972    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:30.086511 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.086474    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"]
Nov 04 01:28:30.091288 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.091197    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kw49c"]
Nov 04 01:28:30.199047 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.199010    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"]
Nov 04 01:28:30.202591 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.202555    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-c563ee-openshift-default-7588b46465-kh2xf"]
Nov 04 01:28:30.248731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.248649    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:30.624469 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:30.624436    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:31.475934 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:31.475906    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" path="/var/lib/kubelet/pods/3e582e99-2e83-43a3-b470-5929d857ffe8/volumes"
Nov 04 01:28:31.476433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:31.476418    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" path="/var/lib/kubelet/pods/8ace3819-81cc-4578-9d47-108614bb3cf7/volumes"
Nov 04 01:28:31.476902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:31.476888    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" path="/var/lib/kubelet/pods/c0c47b89-8d4d-425c-aada-e65032ca072a/volumes"
Nov 04 01:28:31.477658 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:31.477644    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" path="/var/lib/kubelet/pods/c16b56ea-40c6-4e0a-a895-67422da97231/volumes"
Nov 04 01:28:31.478151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:31.478137    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" path="/var/lib/kubelet/pods/db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1/volumes"
Nov 04 01:28:31.858928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:31.858834    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:32.086038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:32.086004    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:32.271135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:32.271074    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:33.147070 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:33.146999    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:35.434842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:35.434307    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:36.912214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:36.912161    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:37.122924 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:37.122840    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:38.637573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:38.637535    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:38.845998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:38.845968    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:38.997454 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:38.997379    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"]
Nov 04 01:28:38.997614 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:38.997591    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy" containerID="cri-o://9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4" gracePeriod=30
Nov 04 01:28:39.047441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:39.047410    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"]
Nov 04 01:28:39.047675 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:39.047642    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" containerID="cri-o://5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c" gracePeriod=30
Nov 04 01:28:39.050647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:39.050621    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"]
Nov 04 01:28:39.050868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:39.050835    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" containerID="cri-o://f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41" gracePeriod=30
Nov 04 01:28:39.068607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:39.068562    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:40.751579 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:40.751511    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:43.162530 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:43.162484    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:28:43.163173 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:43.162571    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:28:43.261742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:43.261707    4434 patch_prober.go:29] interesting pod/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:28:43.261923 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:43.261760    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:28:43.454480 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:43.454387    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:43.690936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:43.690899    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:44.429493 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.429464    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:28:44.476035 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476010    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476063    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476098    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476268 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476200    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476350 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476337    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476387    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476461    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476500    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.476586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.476539    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert\") pod \"17bbbd33-37ab-49ba-944e-325ea2f98749\" (UID: \"17bbbd33-37ab-49ba-944e-325ea2f98749\") "
Nov 04 01:28:44.477038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477014    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.477191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477034    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.477295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477194    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.477295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477206    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.477413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477359    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.477413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477387    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.477505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.477473    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:44.478441 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.478402    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.478545 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.478489    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "kube-api-access-hmg96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:44.479000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.478979    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:44.479569 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.479550    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token" pod "17bbbd33-37ab-49ba-944e-325ea2f98749" (UID: "17bbbd33-37ab-49ba-944e-325ea2f98749"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:44.484588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.484566    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:28:44.486824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.486808    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:28:44.578558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578471    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.578558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578530    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.578832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578602    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.578832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578641    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.578832 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578765    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.579031 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578884    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579031 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578935    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579031 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.578997    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579052    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579120    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.579194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579170    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579482 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579231    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.579482 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579270    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.579613 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579549    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.579613 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579609    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token\") pod \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\" (UID: \"5d52d14d-2ccb-407f-903b-fcc955a84f8d\") "
Nov 04 01:28:44.579745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579708    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579800 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579752    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.579800 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579789    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy\") pod \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\" (UID: \"e58fd0fd-8758-4cae-ba36-96ecc6989b60\") "
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580815    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580848    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17bbbd33-37ab-49ba-944e-325ea2f98749-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580867    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580881    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmg96\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-kube-api-access-hmg96\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580896    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580912    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.581610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580926    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17bbbd33-37ab-49ba-944e-325ea2f98749-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.583265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579782    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579815    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.579968    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580014    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580291    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580479    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580782    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.580811    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.581520    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.581697    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.583455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.582950    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:44.587869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.587842    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.588333 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.588301    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:44.588333 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.588321    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:44.588497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.588384    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:44.588558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.588533    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:44.588824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.588785    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn" pod "e58fd0fd-8758-4cae-ba36-96ecc6989b60" (UID: "e58fd0fd-8758-4cae-ba36-96ecc6989b60"). InnerVolumeSpecName "kube-api-access-sqqwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:44.590081 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.590056    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv" pod "5d52d14d-2ccb-407f-903b-fcc955a84f8d" (UID: "5d52d14d-2ccb-407f-903b-fcc955a84f8d"). InnerVolumeSpecName "kube-api-access-lc6dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:44.682834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682794    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.682834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682829    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682845    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682860    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682874    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682887    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682901    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682914    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682928    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lc6dv\" (UniqueName: \"kubernetes.io/projected/5d52d14d-2ccb-407f-903b-fcc955a84f8d-kube-api-access-lc6dv\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682943    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682957    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqqwn\" (UniqueName: \"kubernetes.io/projected/e58fd0fd-8758-4cae-ba36-96ecc6989b60-kube-api-access-sqqwn\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682974    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.682986    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.683000    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.683013    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.683025    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e58fd0fd-8758-4cae-ba36-96ecc6989b60-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.683040    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:44.683087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:44.683053    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d52d14d-2ccb-407f-903b-fcc955a84f8d-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:45.069038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.068998    4434 generic.go:356] "Generic (PLEG): container finished" podID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerID="5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c" exitCode=0
Nov 04 01:28:45.069236 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.069034    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" event={"ID":"e58fd0fd-8758-4cae-ba36-96ecc6989b60","Type":"ContainerDied","Data":"5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c"}
Nov 04 01:28:45.069236 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.069075    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"
Nov 04 01:28:45.069236 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.069101    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm" event={"ID":"e58fd0fd-8758-4cae-ba36-96ecc6989b60","Type":"ContainerDied","Data":"d85a6b758bdcdd13921d33ed6b7218e249886472b9d58af207bffb7356b4f659"}
Nov 04 01:28:45.069236 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.069131    4434 scope.go:122] "RemoveContainer" containerID="5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c"
Nov 04 01:28:45.070784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.070758    4434 generic.go:356] "Generic (PLEG): container finished" podID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerID="f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41" exitCode=0
Nov 04 01:28:45.070912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.070815    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"
Nov 04 01:28:45.070912 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.070850    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" event={"ID":"5d52d14d-2ccb-407f-903b-fcc955a84f8d","Type":"ContainerDied","Data":"f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41"}
Nov 04 01:28:45.071013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.070938    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8" event={"ID":"5d52d14d-2ccb-407f-903b-fcc955a84f8d","Type":"ContainerDied","Data":"4d63370941f6dd5ab0d3a9aa4b0b6de353e5ac45bd6dceb465f8b81feed60e52"}
Nov 04 01:28:45.074085 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.074062    4434 generic.go:356] "Generic (PLEG): container finished" podID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerID="9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4" exitCode=0
Nov 04 01:28:45.074184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.074112    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" event={"ID":"17bbbd33-37ab-49ba-944e-325ea2f98749","Type":"ContainerDied","Data":"9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4"}
Nov 04 01:28:45.074184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.074119    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"
Nov 04 01:28:45.074184 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.074153    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v" event={"ID":"17bbbd33-37ab-49ba-944e-325ea2f98749","Type":"ContainerDied","Data":"6607ac15a7872b49935f86f4a66b996f9a60b065516902667b498b9c9b627788"}
Nov 04 01:28:45.082799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.082781    4434 scope.go:122] "RemoveContainer" containerID="5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c"
Nov 04 01:28:45.083065 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:45.083046    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c\": container with ID starting with 5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c not found: ID does not exist" containerID="5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c"
Nov 04 01:28:45.083116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.083078    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c"} err="failed to get container status \"5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c\": rpc error: code = NotFound desc = could not find container \"5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c\": container with ID starting with 5d6b59b507de96890917878a0f53513686155c7fa00ca58d9dabcbcd7aeaeb4c not found: ID does not exist"
Nov 04 01:28:45.083116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.083097    4434 scope.go:122] "RemoveContainer" containerID="f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41"
Nov 04 01:28:45.092289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.092269    4434 scope.go:122] "RemoveContainer" containerID="f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41"
Nov 04 01:28:45.092583 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:45.092564    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41\": container with ID starting with f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41 not found: ID does not exist" containerID="f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41"
Nov 04 01:28:45.092660 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.092594    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41"} err="failed to get container status \"f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41\": rpc error: code = NotFound desc = could not find container \"f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41\": container with ID starting with f6f7851449ad5ce6d0f502b357d4b0d9782d518b5ef22411bfa177595964ed41 not found: ID does not exist"
Nov 04 01:28:45.092660 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.092615    4434 scope.go:122] "RemoveContainer" containerID="9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4"
Nov 04 01:28:45.103080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.103064    4434 scope.go:122] "RemoveContainer" containerID="9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4"
Nov 04 01:28:45.103351 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:45.103332    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4\": container with ID starting with 9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4 not found: ID does not exist" containerID="9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4"
Nov 04 01:28:45.103419 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.103357    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4"} err="failed to get container status \"9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4\": rpc error: code = NotFound desc = could not find container \"9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4\": container with ID starting with 9fa207d919b084afa220312301529dfc0d66a0a0d5f7da74d3ab77c4adaf6ac4 not found: ID does not exist"
Nov 04 01:28:45.238424 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.238388    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"]
Nov 04 01:28:45.243551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.243527    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-vvgxm"]
Nov 04 01:28:45.359567 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.359453    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"]
Nov 04 01:28:45.362443 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.362413    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-5cbz8"]
Nov 04 01:28:45.479393 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.479350    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" path="/var/lib/kubelet/pods/5d52d14d-2ccb-407f-903b-fcc955a84f8d/volumes"
Nov 04 01:28:45.480107 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.480087    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" path="/var/lib/kubelet/pods/e58fd0fd-8758-4cae-ba36-96ecc6989b60/volumes"
Nov 04 01:28:45.480495 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.480478    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"]
Nov 04 01:28:45.480532 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:45.480521    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-2ee045-openshift-default-65756bfbc7-mqb8v"]
Nov 04 01:28:46.206824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:46.206783    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:46.375491 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:46.375446    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:47.476063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:47.476025    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" path="/var/lib/kubelet/pods/17bbbd33-37ab-49ba-944e-325ea2f98749/volumes"
Nov 04 01:28:50.701806 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:50.701769    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:51.899288 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:28:51.899243    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:28:52.026558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:52.026515    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"]
Nov 04 01:28:52.026765 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:52.026743    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy" containerID="cri-o://2e6566f88d286468085f542e9bf52f405d4b1d67acbced7394431557ea9f1006" gracePeriod=30
Nov 04 01:28:52.226008 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:52.225907    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:52.420487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:52.420451    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:52.647624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:52.647590    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:52.847095 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:52.847061    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:53.077112 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:53.077030    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:53.268066 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:53.268024    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:57.156762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.156728    4434 generic.go:356] "Generic (PLEG): container finished" podID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerID="2e6566f88d286468085f542e9bf52f405d4b1d67acbced7394431557ea9f1006" exitCode=0
Nov 04 01:28:57.157087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.156795    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" event={"ID":"04c9381e-e7b1-4579-8b32-d32ad9add16e","Type":"ContainerDied","Data":"2e6566f88d286468085f542e9bf52f405d4b1d67acbced7394431557ea9f1006"}
Nov 04 01:28:57.296812 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.296754    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:28:57.371899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.371862    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.371905    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.371990    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372065    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372185 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372159    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:57.372296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372170    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372246    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:57.372296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372289    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372325    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372378    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:57.372457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372394    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372457 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372411    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket\") pod \"04c9381e-e7b1-4579-8b32-d32ad9add16e\" (UID: \"04c9381e-e7b1-4579-8b32-d32ad9add16e\") "
Nov 04 01:28:57.372798 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372774    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:57.372951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.372779    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:28:57.373199 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.373175    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.373300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.373203    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.373300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.373219    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.373300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.373234    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04c9381e-e7b1-4579-8b32-d32ad9add16e-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.373300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.373249    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.374400 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.374374    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:28:57.374494 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.374446    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:28:57.374601 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.374587    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:57.374711 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.374692    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf" pod "04c9381e-e7b1-4579-8b32-d32ad9add16e" (UID: "04c9381e-e7b1-4579-8b32-d32ad9add16e"). InnerVolumeSpecName "kube-api-access-lxlhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:28:57.474895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.474860    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.474895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.474895    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.475182 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.474910    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04c9381e-e7b1-4579-8b32-d32ad9add16e-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:57.475182 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:57.474924    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxlhf\" (UniqueName: \"kubernetes.io/projected/04c9381e-e7b1-4579-8b32-d32ad9add16e-kube-api-access-lxlhf\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:28:58.164609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.164567    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl" event={"ID":"04c9381e-e7b1-4579-8b32-d32ad9add16e","Type":"ContainerDied","Data":"52f1147c3a6cccc31cd898b4b3494e31bda018e7aff37479736c412dcca75ccd"}
Nov 04 01:28:58.164609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.164607    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"
Nov 04 01:28:58.165075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.164646    4434 scope.go:122] "RemoveContainer" containerID="2e6566f88d286468085f542e9bf52f405d4b1d67acbced7394431557ea9f1006"
Nov 04 01:28:58.250777 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.250736    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:58.393342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.393309    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"]
Nov 04 01:28:58.440174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.440091    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:58.443443 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.443417    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-wlwhl"]
Nov 04 01:28:58.991689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:58.990653    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:59.190102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:59.190050    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:59.358979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:59.358877    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:28:59.480856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:59.479393    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" path="/var/lib/kubelet/pods/04c9381e-e7b1-4579-8b32-d32ad9add16e/volumes"
Nov 04 01:28:59.667868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:28:59.667831    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:00.490952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:00.490896    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:00.669705 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:00.669645    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:00.863731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:00.863621    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:01.770299 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:01.770246    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:03.170147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:03.170112    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:03.497637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:03.497554    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:03.740150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:03.740112    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:03.949957 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:03.949910    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:06.290181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:06.290147    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:06.596552 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:06.596464    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:06.710969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:06.710937    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"]
Nov 04 01:29:06.761254 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:06.761223    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"]
Nov 04 01:29:06.764169 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:06.764144    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"]
Nov 04 01:29:06.899716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:06.899683    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:08.214508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:08.214479    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:08.404995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:08.404963    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:09.102732 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:09.102695    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:09.290144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:09.290114    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:09.471448 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:09.471416    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:14.207259 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:14.207213    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:14.674874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:14.674836    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:14.871873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:14.871840    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:19.337698 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:19.337647    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:21.732223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:21.732188    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main" containerID="cri-o://4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002" gracePeriod=45
Nov 04 01:29:21.783302 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:21.783262    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main" containerID="cri-o://59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8" gracePeriod=45
Nov 04 01:29:21.783511 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:21.783494    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main" containerID="cri-o://220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7" gracePeriod=45
Nov 04 01:29:21.907013 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:29:21.906970    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:29:22.067327 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.066920    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:29:22.102139 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.102111    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:29:22.104505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.104484    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:29:22.131745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.131721    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm\") pod \"fde8002a-e824-4850-9581-1f7d62bd7a59\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") "
Nov 04 01:29:22.131870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.131790    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs\" (UniqueName: \"kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs\") pod \"fde8002a-e824-4850-9581-1f7d62bd7a59\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") "
Nov 04 01:29:22.131998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.131974    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home\") pod \"fde8002a-e824-4850-9581-1f7d62bd7a59\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") "
Nov 04 01:29:22.132140 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132028    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t\" (UniqueName: \"kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t\") pod \"fde8002a-e824-4850-9581-1f7d62bd7a59\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") "
Nov 04 01:29:22.132140 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132112    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache\") pod \"fde8002a-e824-4850-9581-1f7d62bd7a59\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") "
Nov 04 01:29:22.132244 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132198    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home" pod "fde8002a-e824-4850-9581-1f7d62bd7a59" (UID: "fde8002a-e824-4850-9581-1f7d62bd7a59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.132244 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132217    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir\") pod \"fde8002a-e824-4850-9581-1f7d62bd7a59\" (UID: \"fde8002a-e824-4850-9581-1f7d62bd7a59\") "
Nov 04 01:29:22.132434 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132411    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache" pod "fde8002a-e824-4850-9581-1f7d62bd7a59" (UID: "fde8002a-e824-4850-9581-1f7d62bd7a59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.132506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132486    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir" pod "fde8002a-e824-4850-9581-1f7d62bd7a59" (UID: "fde8002a-e824-4850-9581-1f7d62bd7a59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.132796 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132778    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.132796 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132797    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.132939 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.132824    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.134070 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.134046    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t" pod "fde8002a-e824-4850-9581-1f7d62bd7a59" (UID: "fde8002a-e824-4850-9581-1f7d62bd7a59"). InnerVolumeSpecName "kube-api-access-fqz9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:22.134144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.134075    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm" pod "fde8002a-e824-4850-9581-1f7d62bd7a59" (UID: "fde8002a-e824-4850-9581-1f7d62bd7a59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.134144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.134075    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs" pod "fde8002a-e824-4850-9581-1f7d62bd7a59" (UID: "fde8002a-e824-4850-9581-1f7d62bd7a59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:22.233372 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233308    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home\") pod \"c30d109b-c451-4de5-90e4-9a1a670762d5\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") "
Nov 04 01:29:22.233372 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233376    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home\") pod \"2fccb086-7049-42d2-818e-2c16e8492cee\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233418    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm\") pod \"c30d109b-c451-4de5-90e4-9a1a670762d5\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233463    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache\") pod \"2fccb086-7049-42d2-818e-2c16e8492cee\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233493    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf\" (UniqueName: \"kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf\") pod \"c30d109b-c451-4de5-90e4-9a1a670762d5\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233515    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache\") pod \"c30d109b-c451-4de5-90e4-9a1a670762d5\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233560    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr\" (UniqueName: \"kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr\") pod \"2fccb086-7049-42d2-818e-2c16e8492cee\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233585    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs\" (UniqueName: \"kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs\") pod \"c30d109b-c451-4de5-90e4-9a1a670762d5\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") "
Nov 04 01:29:22.233626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233628    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs\" (UniqueName: \"kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs\") pod \"2fccb086-7049-42d2-818e-2c16e8492cee\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") "
Nov 04 01:29:22.234040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233641    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home" pod "c30d109b-c451-4de5-90e4-9a1a670762d5" (UID: "c30d109b-c451-4de5-90e4-9a1a670762d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.234040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233758    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir\") pod \"c30d109b-c451-4de5-90e4-9a1a670762d5\" (UID: \"c30d109b-c451-4de5-90e4-9a1a670762d5\") "
Nov 04 01:29:22.234040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233853    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir\") pod \"2fccb086-7049-42d2-818e-2c16e8492cee\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") "
Nov 04 01:29:22.234040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233909    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache" pod "c30d109b-c451-4de5-90e4-9a1a670762d5" (UID: "c30d109b-c451-4de5-90e4-9a1a670762d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.234040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.233988    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm\") pod \"2fccb086-7049-42d2-818e-2c16e8492cee\" (UID: \"2fccb086-7049-42d2-818e-2c16e8492cee\") "
Nov 04 01:29:22.234301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234039    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home" pod "2fccb086-7049-42d2-818e-2c16e8492cee" (UID: "2fccb086-7049-42d2-818e-2c16e8492cee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.234301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234244    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir" pod "c30d109b-c451-4de5-90e4-9a1a670762d5" (UID: "c30d109b-c451-4de5-90e4-9a1a670762d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234716    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqz9t\" (UniqueName: \"kubernetes.io/projected/fde8002a-e824-4850-9581-1f7d62bd7a59-kube-api-access-fqz9t\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234731    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache" pod "2fccb086-7049-42d2-818e-2c16e8492cee" (UID: "2fccb086-7049-42d2-818e-2c16e8492cee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234742    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234757    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234771    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234786    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.234804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234800    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde8002a-e824-4850-9581-1f7d62bd7a59-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.235227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234812    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde8002a-e824-4850-9581-1f7d62bd7a59-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.235227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.234784    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir" pod "2fccb086-7049-42d2-818e-2c16e8492cee" (UID: "2fccb086-7049-42d2-818e-2c16e8492cee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.237467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.237438    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs" pod "c30d109b-c451-4de5-90e4-9a1a670762d5" (UID: "c30d109b-c451-4de5-90e4-9a1a670762d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:22.237598 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.237575    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf" pod "c30d109b-c451-4de5-90e4-9a1a670762d5" (UID: "c30d109b-c451-4de5-90e4-9a1a670762d5"). InnerVolumeSpecName "kube-api-access-7x6lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:22.237701 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.237614    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr" pod "2fccb086-7049-42d2-818e-2c16e8492cee" (UID: "2fccb086-7049-42d2-818e-2c16e8492cee"). InnerVolumeSpecName "kube-api-access-5vrrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:22.237986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.237967    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm" pod "2fccb086-7049-42d2-818e-2c16e8492cee" (UID: "2fccb086-7049-42d2-818e-2c16e8492cee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.238094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.238076    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm" pod "c30d109b-c451-4de5-90e4-9a1a670762d5" (UID: "c30d109b-c451-4de5-90e4-9a1a670762d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:29:22.239301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.239267    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs" pod "2fccb086-7049-42d2-818e-2c16e8492cee" (UID: "2fccb086-7049-42d2-818e-2c16e8492cee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:22.322304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.321206    4434 generic.go:356] "Generic (PLEG): container finished" podID="2fccb086-7049-42d2-818e-2c16e8492cee" containerID="220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7" exitCode=0
Nov 04 01:29:22.322304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.321309    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" event={"ID":"2fccb086-7049-42d2-818e-2c16e8492cee","Type":"ContainerDied","Data":"220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7"}
Nov 04 01:29:22.322304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.321347    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs" event={"ID":"2fccb086-7049-42d2-818e-2c16e8492cee","Type":"ContainerDied","Data":"37430d26335498d92219eaebc3545a94c2c5b2818fe26bc9578cd1a182e09b06"}
Nov 04 01:29:22.322304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.321370    4434 scope.go:122] "RemoveContainer" containerID="220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7"
Nov 04 01:29:22.322304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.321519    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"
Nov 04 01:29:22.331436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.330793    4434 generic.go:356] "Generic (PLEG): container finished" podID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerID="4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002" exitCode=0
Nov 04 01:29:22.331436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.330950    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" event={"ID":"fde8002a-e824-4850-9581-1f7d62bd7a59","Type":"ContainerDied","Data":"4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002"}
Nov 04 01:29:22.331436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.330988    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b" event={"ID":"fde8002a-e824-4850-9581-1f7d62bd7a59","Type":"ContainerDied","Data":"e9c123b68d460eecdbc349313e0a2e53551e2d9fa5bd51c6051200d1b7b7657f"}
Nov 04 01:29:22.331436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.331064    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"
Nov 04 01:29:22.336942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.336913    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.336942 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.336943    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.337136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.336955    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c30d109b-c451-4de5-90e4-9a1a670762d5-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.337136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.336968    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fccb086-7049-42d2-818e-2c16e8492cee-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.337136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.336982    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x6lf\" (UniqueName: \"kubernetes.io/projected/c30d109b-c451-4de5-90e4-9a1a670762d5-kube-api-access-7x6lf\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.337136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.336996    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vrrr\" (UniqueName: \"kubernetes.io/projected/2fccb086-7049-42d2-818e-2c16e8492cee-kube-api-access-5vrrr\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.337136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.337010    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c30d109b-c451-4de5-90e4-9a1a670762d5-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.337136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.337032    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fccb086-7049-42d2-818e-2c16e8492cee-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:22.338063 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.338033    4434 generic.go:356] "Generic (PLEG): container finished" podID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerID="59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8" exitCode=0
Nov 04 01:29:22.338166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.338090    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"
Nov 04 01:29:22.338166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.338130    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" event={"ID":"c30d109b-c451-4de5-90e4-9a1a670762d5","Type":"ContainerDied","Data":"59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8"}
Nov 04 01:29:22.338278 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.338165    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk" event={"ID":"c30d109b-c451-4de5-90e4-9a1a670762d5","Type":"ContainerDied","Data":"f66bcfef057acec90869f0ad4e06881aa12751c28189a2cbdbb7ab8d90c7fedc"}
Nov 04 01:29:22.339326 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.339160    4434 scope.go:122] "RemoveContainer" containerID="220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7"
Nov 04 01:29:22.339516 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:29:22.339478    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7\": container with ID starting with 220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7 not found: ID does not exist" containerID="220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7"
Nov 04 01:29:22.339589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.339517    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7"} err="failed to get container status \"220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7\": rpc error: code = NotFound desc = could not find container \"220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7\": container with ID starting with 220fa540a9ce9455b4500f9a69a0903dcde8fa2c1dacc61df3e569aab7b88ff7 not found: ID does not exist"
Nov 04 01:29:22.339589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.339540    4434 scope.go:122] "RemoveContainer" containerID="4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002"
Nov 04 01:29:22.352452 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.352369    4434 scope.go:122] "RemoveContainer" containerID="4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002"
Nov 04 01:29:22.352714 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:29:22.352637    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002\": container with ID starting with 4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002 not found: ID does not exist" containerID="4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002"
Nov 04 01:29:22.352811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.352704    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002"} err="failed to get container status \"4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002\": rpc error: code = NotFound desc = could not find container \"4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002\": container with ID starting with 4a69ce5b70d23a1bce693b3ae13565a3bc0dedf17d7412a2e038cf0bd1588002 not found: ID does not exist"
Nov 04 01:29:22.352811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.352727    4434 scope.go:122] "RemoveContainer" containerID="59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8"
Nov 04 01:29:22.364264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.364243    4434 scope.go:122] "RemoveContainer" containerID="59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8"
Nov 04 01:29:22.364635 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:29:22.364609    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8\": container with ID starting with 59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8 not found: ID does not exist" containerID="59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8"
Nov 04 01:29:22.364736 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.364644    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8"} err="failed to get container status \"59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8\": rpc error: code = NotFound desc = could not find container \"59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8\": container with ID starting with 59d702974e8b22b810d4d20bb9b97ef30deea1a1f23d728701771817d1aec8e8 not found: ID does not exist"
Nov 04 01:29:22.564182 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.564122    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"]
Nov 04 01:29:22.567966 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.567933    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/premium-w4-e65b23-kserve-59f874f59f-wknwk"]
Nov 04 01:29:22.678723 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.678692    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"]
Nov 04 01:29:22.683075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.683052    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/simulator-w4-e65b23-kserve-6666544c66-6s6zs"]
Nov 04 01:29:22.794241 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.794202    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"]
Nov 04 01:29:22.798781 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:22.798747    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e65b23/distinct-w4-e65b23-kserve-746968d85d-hmp4b"]
Nov 04 01:29:23.475775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:23.475703    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" path="/var/lib/kubelet/pods/2fccb086-7049-42d2-818e-2c16e8492cee/volumes"
Nov 04 01:29:23.476162 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:23.476148    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" path="/var/lib/kubelet/pods/c30d109b-c451-4de5-90e4-9a1a670762d5/volumes"
Nov 04 01:29:23.476551 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:23.476539    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" path="/var/lib/kubelet/pods/fde8002a-e824-4850-9581-1f7d62bd7a59/volumes"
Nov 04 01:29:24.285808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:24.285774    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:24.696001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:24.695966    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:24.902970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:24.902939    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:29.627941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:29.627907    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:34.351845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:34.351807    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:34.587633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:34.587594    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:39.726704 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:39.726651    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:44.175879 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:44.175794    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:44.369510 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:44.369478    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:45.315066 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:45.315023    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:45.845941 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:45.845897    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:49.719181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:49.719144    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"]
Nov 04 01:29:49.719641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:49.719380    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" podUID="3439c0be-bdb1-449e-a12c-d7553ec0d88e" containerName="maas-api" containerID="cri-o://713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae" gracePeriod=30
Nov 04 01:29:49.870716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:49.870653    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:50.048947 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.048918    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:29:50.219350 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.219318    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca\") pod \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") "
Nov 04 01:29:50.219573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.219362    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\") pod \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") "
Nov 04 01:29:50.219573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.219380    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs\") pod \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") "
Nov 04 01:29:50.219573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.219413    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d\" (UniqueName: \"kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d\") pod \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\" (UID: \"3439c0be-bdb1-449e-a12c-d7553ec0d88e\") "
Nov 04 01:29:50.219876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.219818    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca" pod "3439c0be-bdb1-449e-a12c-d7553ec0d88e" (UID: "3439c0be-bdb1-449e-a12c-d7553ec0d88e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:29:50.220161 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.220135    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3439c0be-bdb1-449e-a12c-d7553ec0d88e-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:50.221626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.221597    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls" pod "3439c0be-bdb1-449e-a12c-d7553ec0d88e" (UID: "3439c0be-bdb1-449e-a12c-d7553ec0d88e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:50.221746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.221618    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs" pod "3439c0be-bdb1-449e-a12c-d7553ec0d88e" (UID: "3439c0be-bdb1-449e-a12c-d7553ec0d88e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:29:50.221746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.221598    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d" pod "3439c0be-bdb1-449e-a12c-d7553ec0d88e" (UID: "3439c0be-bdb1-449e-a12c-d7553ec0d88e"). InnerVolumeSpecName "kube-api-access-xbt5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:29:50.321745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.321624    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:50.321745 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.321711    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3439c0be-bdb1-449e-a12c-d7553ec0d88e-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:50.321949 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.321766    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbt5d\" (UniqueName: \"kubernetes.io/projected/3439c0be-bdb1-449e-a12c-d7553ec0d88e-kube-api-access-xbt5d\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:29:50.532952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.532914    4434 generic.go:356] "Generic (PLEG): container finished" podID="3439c0be-bdb1-449e-a12c-d7553ec0d88e" containerID="713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae" exitCode=0
Nov 04 01:29:50.533125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.532959    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"
Nov 04 01:29:50.533125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.532994    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" event={"ID":"3439c0be-bdb1-449e-a12c-d7553ec0d88e","Type":"ContainerDied","Data":"713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae"}
Nov 04 01:29:50.533125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.533045    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg" event={"ID":"3439c0be-bdb1-449e-a12c-d7553ec0d88e","Type":"ContainerDied","Data":"fbe1441a6339635191a04ef3a1ed62479207aad384e6d016bb52cfc43b735749"}
Nov 04 01:29:50.533125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.533066    4434 scope.go:122] "RemoveContainer" containerID="713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae"
Nov 04 01:29:50.543116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.543094    4434 scope.go:122] "RemoveContainer" containerID="713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae"
Nov 04 01:29:50.543366 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:29:50.543347    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae\": container with ID starting with 713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae not found: ID does not exist" containerID="713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae"
Nov 04 01:29:50.543434 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.543379    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae"} err="failed to get container status \"713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae\": rpc error: code = NotFound desc = could not find container \"713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae\": container with ID starting with 713e2ab474393fb0117de119ddabd92c751c08eedb7029fb90e95f3d46feeaae not found: ID does not exist"
Nov 04 01:29:50.749314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.748501    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"]
Nov 04 01:29:50.763090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:50.763054    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-c563ee-8456f5dd45-kxzlg"]
Nov 04 01:29:51.450455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:51.450426    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:51.474245 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:51.474216    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3439c0be-bdb1-449e-a12c-d7553ec0d88e" path="/var/lib/kubelet/pods/3439c0be-bdb1-449e-a12c-d7553ec0d88e/volumes"
Nov 04 01:29:51.627511 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:51.627476    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:51.804588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:51.804500    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:51.899437 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:29:51.899401    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:29:52.465708 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:52.465653    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:57.660930 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:57.660894    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:57.934934 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:57.934855    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:58.122200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:58.122156    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:58.874698 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:58.874617    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:59.066635 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:59.066599    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:29:59.278267 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:29:59.278094    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:00.275775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.275733    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"]
Nov 04 01:30:00.276304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276289    4434 cpu_manager.go:414] "containerMap: removing container" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main"
Nov 04 01:30:00.276349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276304    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main"
Nov 04 01:30:00.276349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276317    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy"
Nov 04 01:30:00.276349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276324    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy"
Nov 04 01:30:00.276349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276338    4434 cpu_manager.go:414] "containerMap: removing container" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerName="payload-pre-processing"
Nov 04 01:30:00.276349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276343    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerName="payload-pre-processing"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276351    4434 cpu_manager.go:414] "containerMap: removing container" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276356    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276367    4434 cpu_manager.go:414] "containerMap: removing container" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276372    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276379    4434 cpu_manager.go:414] "containerMap: removing container" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276383    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276394    4434 cpu_manager.go:414] "containerMap: removing container" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276399    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276405    4434 cpu_manager.go:414] "containerMap: removing container" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276410    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276418    4434 cpu_manager.go:414] "containerMap: removing container" podUID="3439c0be-bdb1-449e-a12c-d7553ec0d88e" containerName="maas-api"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276422    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3439c0be-bdb1-449e-a12c-d7553ec0d88e" containerName="maas-api"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276428    4434 cpu_manager.go:414] "containerMap: removing container" podUID="d14b7260-edaa-4fa8-a103-5fa1ec753886" containerName="payload-pre-processing"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276433    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d14b7260-edaa-4fa8-a103-5fa1ec753886" containerName="payload-pre-processing"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276442    4434 cpu_manager.go:414] "containerMap: removing container" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276447    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276457    4434 cpu_manager.go:414] "containerMap: removing container" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276461    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276472    4434 cpu_manager.go:414] "containerMap: removing container" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276476    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276485    4434 cpu_manager.go:414] "containerMap: removing container" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276490    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276497    4434 cpu_manager.go:414] "containerMap: removing container" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276501    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276507    4434 cpu_manager.go:414] "containerMap: removing container" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy"
Nov 04 01:30:00.276503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276512    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276528    4434 cpu_manager.go:414] "containerMap: removing container" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276533    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276605    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde8002a-e824-4850-9581-1f7d62bd7a59" containerName="main"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276614    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e58fd0fd-8758-4cae-ba36-96ecc6989b60" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276621    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fc5756b-93da-49c1-af2b-e0f5e341609d" containerName="payload-pre-processing"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276628    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fff6230-5d60-49e0-b116-3f060d15a052" containerName="main"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276633    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ace3819-81cc-4578-9d47-108614bb3cf7" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276639    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="04c9381e-e7b1-4579-8b32-d32ad9add16e" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276646    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d52d14d-2ccb-407f-903b-fcc955a84f8d" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276651    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0c47b89-8d4d-425c-aada-e65032ca072a" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276657    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="3439c0be-bdb1-449e-a12c-d7553ec0d88e" containerName="maas-api"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276678    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="d14b7260-edaa-4fa8-a103-5fa1ec753886" containerName="payload-pre-processing"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276685    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="17bbbd33-37ab-49ba-944e-325ea2f98749" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276690    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="c16b56ea-40c6-4e0a-a895-67422da97231" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276697    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="db3a9c47-d27c-4d4c-ab61-5c46aaac8aa1" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276703    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fccb086-7049-42d2-818e-2c16e8492cee" containerName="main"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276709    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a3ab8da-14e4-4a0f-8300-88850af83969" containerName="main"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276713    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e582e99-2e83-43a3-b470-5929d857ffe8" containerName="istio-proxy"
Nov 04 01:30:00.277291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.276719    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="c30d109b-c451-4de5-90e4-9a1a670762d5" containerName="main"
Nov 04 01:30:00.280230 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.280213    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.320892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.320849    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"]
Nov 04 01:30:00.324556 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.324533    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.342454 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.342419    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"]
Nov 04 01:30:00.348133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.347554    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.355011 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.354652    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.355165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.355066    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wlnk\" (UniqueName: \"kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk\") pod \"maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.355165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.355128    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r7dv\" (UniqueName: \"kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv\") pod \"maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.355311 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.355283    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.366400 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.366372    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qb9zm\""
Nov 04 01:30:00.369370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.369347    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"]
Nov 04 01:30:00.394022 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.393987    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"]
Nov 04 01:30:00.410967 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.410936    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"]
Nov 04 01:30:00.456399 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.456363    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.456618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.456440    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wlnk\" (UniqueName: \"kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk\") pod \"maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.456618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.456493    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r7dv\" (UniqueName: \"kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv\") pod \"maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.456618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.456574    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.456849 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.456710    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.456849 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.456738    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btz7n\" (UniqueName: \"kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.457292 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.457267    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.457380 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.457359    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.522258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.522226    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r7dv\" (UniqueName: \"kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv\") pod \"maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.522258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.522250    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wlnk\" (UniqueName: \"kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk\") pod \"maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.558158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.558031    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.558158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.558080    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btz7n\" (UniqueName: \"kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.558860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.558834    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.596443 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.596416    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:00.619024 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.618996    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btz7n\" (UniqueName: \"kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.638946 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.638653    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:00.664695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.664630    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:00.979340 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:00.979302    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"]
Nov 04 01:30:00.982024 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:30:00.981992    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10fad34_9218_48f1_863f_05b54184b28f.slice/crio-2a085ea482596a53225eb7f5a217fbd6243f81a9c10fe40bf7680251d0a5ad95 WatchSource:0}: Error finding container 2a085ea482596a53225eb7f5a217fbd6243f81a9c10fe40bf7680251d0a5ad95: Status 404 returned error can't find the container with id 2a085ea482596a53225eb7f5a217fbd6243f81a9c10fe40bf7680251d0a5ad95
Nov 04 01:30:01.033737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.033699    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"]
Nov 04 01:30:01.036573 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:30:01.036544    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod036e577b_422c_4d64_aad7_8d1a68f36417.slice/crio-c71deb9da13a0d0ec7187df0945211f5dac94d24b431ab4dbebede7a489aafef WatchSource:0}: Error finding container c71deb9da13a0d0ec7187df0945211f5dac94d24b431ab4dbebede7a489aafef: Status 404 returned error can't find the container with id c71deb9da13a0d0ec7187df0945211f5dac94d24b431ab4dbebede7a489aafef
Nov 04 01:30:01.084316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.084291    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"]
Nov 04 01:30:01.087194 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:30:01.087167    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81eda2bb_32a2_48dd_b3d0_9194f3170c1d.slice/crio-fa5c120cbef44d73614d3e5186c904abd983df1507ff525a9c05865a07f1ba45 WatchSource:0}: Error finding container fa5c120cbef44d73614d3e5186c904abd983df1507ff525a9c05865a07f1ba45: Status 404 returned error can't find the container with id fa5c120cbef44d73614d3e5186c904abd983df1507ff525a9c05865a07f1ba45
Nov 04 01:30:01.531906 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.531821    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:01.601352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.601310    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" event={"ID":"e10fad34-9218-48f1-863f-05b54184b28f","Type":"ContainerStarted","Data":"ed696febe375dcc11d88e6d7448629fd936efa898dd5b40a8797057df8a9625c"}
Nov 04 01:30:01.601586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.601376    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" event={"ID":"e10fad34-9218-48f1-863f-05b54184b28f","Type":"ContainerStarted","Data":"2a085ea482596a53225eb7f5a217fbd6243f81a9c10fe40bf7680251d0a5ad95"}
Nov 04 01:30:01.607731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.607660    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" event={"ID":"036e577b-422c-4d64-aad7-8d1a68f36417","Type":"ContainerStarted","Data":"7303442946a429db7935a1ad2fc0d73ab39185df065b90dd5a2d601e8632030d"}
Nov 04 01:30:01.607906 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.607757    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" event={"ID":"036e577b-422c-4d64-aad7-8d1a68f36417","Type":"ContainerStarted","Data":"c71deb9da13a0d0ec7187df0945211f5dac94d24b431ab4dbebede7a489aafef"}
Nov 04 01:30:01.609496 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.609454    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerStarted","Data":"2f6ab0580e7e4c43b1b31df6e24e24db3f375c8dc946d0581d1aa37a18d451ab"}
Nov 04 01:30:01.609623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.609512    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerStarted","Data":"fa5c120cbef44d73614d3e5186c904abd983df1507ff525a9c05865a07f1ba45"}
Nov 04 01:30:01.724089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.724020    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" podStartSLOduration=1.724004124 podStartE2EDuration="1.724004124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:01.722612843 +0000 UTC m=+1970.384752905" watchObservedRunningTime="2026-11-04 01:30:01.724004124 +0000 UTC m=+1970.386144185"
Nov 04 01:30:01.842884 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.842775    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" podStartSLOduration=1.842760001 podStartE2EDuration="1.842760001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:01.840759237 +0000 UTC m=+1970.502899299" watchObservedRunningTime="2026-11-04 01:30:01.842760001 +0000 UTC m=+1970.504900103"
Nov 04 01:30:01.959188 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:01.959130    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" podStartSLOduration=1.959114849 podStartE2EDuration="1.959114849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:30:01.957303856 +0000 UTC m=+1970.619443917" watchObservedRunningTime="2026-11-04 01:30:01.959114849 +0000 UTC m=+1970.621254910"
Nov 04 01:30:02.617605 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.617506    4434 generic.go:356] "Generic (PLEG): container finished" podID="e10fad34-9218-48f1-863f-05b54184b28f" containerID="ed696febe375dcc11d88e6d7448629fd936efa898dd5b40a8797057df8a9625c" exitCode=60
Nov 04 01:30:02.618061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.617590    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" event={"ID":"e10fad34-9218-48f1-863f-05b54184b28f","Type":"ContainerDied","Data":"ed696febe375dcc11d88e6d7448629fd936efa898dd5b40a8797057df8a9625c"}
Nov 04 01:30:02.618061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.617912    4434 scope.go:122] "RemoveContainer" containerID="ed696febe375dcc11d88e6d7448629fd936efa898dd5b40a8797057df8a9625c"
Nov 04 01:30:02.619597 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.619568    4434 generic.go:356] "Generic (PLEG): container finished" podID="036e577b-422c-4d64-aad7-8d1a68f36417" containerID="7303442946a429db7935a1ad2fc0d73ab39185df065b90dd5a2d601e8632030d" exitCode=60
Nov 04 01:30:02.619734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.619657    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" event={"ID":"036e577b-422c-4d64-aad7-8d1a68f36417","Type":"ContainerDied","Data":"7303442946a429db7935a1ad2fc0d73ab39185df065b90dd5a2d601e8632030d"}
Nov 04 01:30:02.619952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.619933    4434 scope.go:122] "RemoveContainer" containerID="7303442946a429db7935a1ad2fc0d73ab39185df065b90dd5a2d601e8632030d"
Nov 04 01:30:02.621348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.621322    4434 generic.go:356] "Generic (PLEG): container finished" podID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerID="2f6ab0580e7e4c43b1b31df6e24e24db3f375c8dc946d0581d1aa37a18d451ab" exitCode=60
Nov 04 01:30:02.621459 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.621369    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerDied","Data":"2f6ab0580e7e4c43b1b31df6e24e24db3f375c8dc946d0581d1aa37a18d451ab"}
Nov 04 01:30:02.621760 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:02.621651    4434 scope.go:122] "RemoveContainer" containerID="2f6ab0580e7e4c43b1b31df6e24e24db3f375c8dc946d0581d1aa37a18d451ab"
Nov 04 01:30:03.630296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.630256    4434 generic.go:356] "Generic (PLEG): container finished" podID="036e577b-422c-4d64-aad7-8d1a68f36417" containerID="fd4201c04c2f8b9a258d1a48f24270a386dc7a1f7c549b03c7d12f37703b1da9" exitCode=60
Nov 04 01:30:03.630818 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.630337    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" event={"ID":"036e577b-422c-4d64-aad7-8d1a68f36417","Type":"ContainerDied","Data":"fd4201c04c2f8b9a258d1a48f24270a386dc7a1f7c549b03c7d12f37703b1da9"}
Nov 04 01:30:03.630818 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.630395    4434 scope.go:122] "RemoveContainer" containerID="7303442946a429db7935a1ad2fc0d73ab39185df065b90dd5a2d601e8632030d"
Nov 04 01:30:03.630818 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.630706    4434 scope.go:122] "RemoveContainer" containerID="fd4201c04c2f8b9a258d1a48f24270a386dc7a1f7c549b03c7d12f37703b1da9"
Nov 04 01:30:03.631087 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:03.631056    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj_odh-ai-gateway-infra(036e577b-422c-4d64-aad7-8d1a68f36417)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" podUID="036e577b-422c-4d64-aad7-8d1a68f36417"
Nov 04 01:30:03.632412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.632389    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerStarted","Data":"49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0"}
Nov 04 01:30:03.634862 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.634829    4434 generic.go:356] "Generic (PLEG): container finished" podID="e10fad34-9218-48f1-863f-05b54184b28f" containerID="57804e175a3c80f47ed67905a86c37eebfb3657805964d4e376c8be7842431ac" exitCode=60
Nov 04 01:30:03.634961 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.634866    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" event={"ID":"e10fad34-9218-48f1-863f-05b54184b28f","Type":"ContainerDied","Data":"57804e175a3c80f47ed67905a86c37eebfb3657805964d4e376c8be7842431ac"}
Nov 04 01:30:03.635132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.635118    4434 scope.go:122] "RemoveContainer" containerID="57804e175a3c80f47ed67905a86c37eebfb3657805964d4e376c8be7842431ac"
Nov 04 01:30:03.635404 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:03.635384    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz_odh-ai-gateway-infra(e10fad34-9218-48f1-863f-05b54184b28f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" podUID="e10fad34-9218-48f1-863f-05b54184b28f"
Nov 04 01:30:03.741891 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.741520    4434 scope.go:122] "RemoveContainer" containerID="ed696febe375dcc11d88e6d7448629fd936efa898dd5b40a8797057df8a9625c"
Nov 04 01:30:03.763275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:03.763242    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:04.647607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:04.647576    4434 scope.go:122] "RemoveContainer" containerID="fd4201c04c2f8b9a258d1a48f24270a386dc7a1f7c549b03c7d12f37703b1da9"
Nov 04 01:30:04.648132 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:04.647962    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj_odh-ai-gateway-infra(036e577b-422c-4d64-aad7-8d1a68f36417)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" podUID="036e577b-422c-4d64-aad7-8d1a68f36417"
Nov 04 01:30:04.649142 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:04.649115    4434 generic.go:356] "Generic (PLEG): container finished" podID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerID="49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0" exitCode=60
Nov 04 01:30:04.649282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:04.649192    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerDied","Data":"49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0"}
Nov 04 01:30:04.649282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:04.649272    4434 scope.go:122] "RemoveContainer" containerID="2f6ab0580e7e4c43b1b31df6e24e24db3f375c8dc946d0581d1aa37a18d451ab"
Nov 04 01:30:04.649448 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:04.649432    4434 scope.go:122] "RemoveContainer" containerID="49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0"
Nov 04 01:30:04.649922 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:04.649820    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt_odh-ai-gateway-infra(81eda2bb-32a2-48dd-b3d0-9194f3170c1d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d"
Nov 04 01:30:04.651496 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:04.651478    4434 scope.go:122] "RemoveContainer" containerID="57804e175a3c80f47ed67905a86c37eebfb3657805964d4e376c8be7842431ac"
Nov 04 01:30:04.651808 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:04.651787    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz_odh-ai-gateway-infra(e10fad34-9218-48f1-863f-05b54184b28f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" podUID="e10fad34-9218-48f1-863f-05b54184b28f"
Nov 04 01:30:05.661580 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:05.661539    4434 scope.go:122] "RemoveContainer" containerID="49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0"
Nov 04 01:30:05.662168 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:05.661942    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt_odh-ai-gateway-infra(81eda2bb-32a2-48dd-b3d0-9194f3170c1d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d"
Nov 04 01:30:06.618644 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:06.618588    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:09.249130 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:09.249089    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:14.607871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:14.607832    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:16.467796 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:16.467761    4434 scope.go:122] "RemoveContainer" containerID="49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0"
Nov 04 01:30:16.468275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:16.467897    4434 scope.go:122] "RemoveContainer" containerID="fd4201c04c2f8b9a258d1a48f24270a386dc7a1f7c549b03c7d12f37703b1da9"
Nov 04 01:30:16.468275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:16.468200    4434 scope.go:122] "RemoveContainer" containerID="57804e175a3c80f47ed67905a86c37eebfb3657805964d4e376c8be7842431ac"
Nov 04 01:30:16.686989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:16.686962    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:17.232686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.229951    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:17.737258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.737215    4434 generic.go:356] "Generic (PLEG): container finished" podID="036e577b-422c-4d64-aad7-8d1a68f36417" containerID="5e7e519e33724f27282f056062a25b721763045d86d981f905094d5b29edaa2f" exitCode=60
Nov 04 01:30:17.737258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.737248    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" event={"ID":"036e577b-422c-4d64-aad7-8d1a68f36417","Type":"ContainerDied","Data":"5e7e519e33724f27282f056062a25b721763045d86d981f905094d5b29edaa2f"}
Nov 04 01:30:17.737803 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.737321    4434 scope.go:122] "RemoveContainer" containerID="fd4201c04c2f8b9a258d1a48f24270a386dc7a1f7c549b03c7d12f37703b1da9"
Nov 04 01:30:17.737868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.737823    4434 scope.go:122] "RemoveContainer" containerID="5e7e519e33724f27282f056062a25b721763045d86d981f905094d5b29edaa2f"
Nov 04 01:30:17.738154 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:17.738129    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj_odh-ai-gateway-infra(036e577b-422c-4d64-aad7-8d1a68f36417)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" podUID="036e577b-422c-4d64-aad7-8d1a68f36417"
Nov 04 01:30:17.739948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.739922    4434 generic.go:356] "Generic (PLEG): container finished" podID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerID="ad13872ae06993f6205f790094174e312979d0717bbef063e6f5e138039fe980" exitCode=60
Nov 04 01:30:17.740062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.740009    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerDied","Data":"ad13872ae06993f6205f790094174e312979d0717bbef063e6f5e138039fe980"}
Nov 04 01:30:17.740420 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.740402    4434 scope.go:122] "RemoveContainer" containerID="ad13872ae06993f6205f790094174e312979d0717bbef063e6f5e138039fe980"
Nov 04 01:30:17.740759 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:17.740739    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt_odh-ai-gateway-infra(81eda2bb-32a2-48dd-b3d0-9194f3170c1d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d"
Nov 04 01:30:17.742552 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.742528    4434 generic.go:356] "Generic (PLEG): container finished" podID="e10fad34-9218-48f1-863f-05b54184b28f" containerID="778b2788dc1069ec8881f5b5aed2fd5869188ec8c6857d4ed0deeb4b44ab71f3" exitCode=60
Nov 04 01:30:17.742648 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.742596    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" event={"ID":"e10fad34-9218-48f1-863f-05b54184b28f","Type":"ContainerDied","Data":"778b2788dc1069ec8881f5b5aed2fd5869188ec8c6857d4ed0deeb4b44ab71f3"}
Nov 04 01:30:17.745526 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.742962    4434 scope.go:122] "RemoveContainer" containerID="778b2788dc1069ec8881f5b5aed2fd5869188ec8c6857d4ed0deeb4b44ab71f3"
Nov 04 01:30:17.745526 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:17.743248    4434 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz_odh-ai-gateway-infra(e10fad34-9218-48f1-863f-05b54184b28f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" podUID="e10fad34-9218-48f1-863f-05b54184b28f"
Nov 04 01:30:17.919040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.910142    4434 scope.go:122] "RemoveContainer" containerID="49acb1b2e490d0d9434bdbf057ed31f291f30ec0b0422071bb19fb37860c38b0"
Nov 04 01:30:17.928863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.927343    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:17.951820 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:17.951690    4434 scope.go:122] "RemoveContainer" containerID="57804e175a3c80f47ed67905a86c37eebfb3657805964d4e376c8be7842431ac"
Nov 04 01:30:18.383498 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:18.383470    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:18.903110 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:18.903076    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"]
Nov 04 01:30:19.052708 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.052678    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"]
Nov 04 01:30:19.061314 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.061290    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:19.141046 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.141014    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"]
Nov 04 01:30:19.173055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.171012    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk\" (UniqueName: \"kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk\") pod \"036e577b-422c-4d64-aad7-8d1a68f36417\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") "
Nov 04 01:30:19.173055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.171246    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca\") pod \"036e577b-422c-4d64-aad7-8d1a68f36417\" (UID: \"036e577b-422c-4d64-aad7-8d1a68f36417\") "
Nov 04 01:30:19.173055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.171923    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca" pod "036e577b-422c-4d64-aad7-8d1a68f36417" (UID: "036e577b-422c-4d64-aad7-8d1a68f36417"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:19.180724 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.174050    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk" pod "036e577b-422c-4d64-aad7-8d1a68f36417" (UID: "036e577b-422c-4d64-aad7-8d1a68f36417"). InnerVolumeSpecName "kube-api-access-4wlnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:19.210891 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.210862    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:19.272370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.272336    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca\") pod \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") "
Nov 04 01:30:19.272545 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.272429    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n\" (UniqueName: \"kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n\") pod \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\" (UID: \"81eda2bb-32a2-48dd-b3d0-9194f3170c1d\") "
Nov 04 01:30:19.272877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.272847    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca" pod "81eda2bb-32a2-48dd-b3d0-9194f3170c1d" (UID: "81eda2bb-32a2-48dd-b3d0-9194f3170c1d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:19.273100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.273078    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wlnk\" (UniqueName: \"kubernetes.io/projected/036e577b-422c-4d64-aad7-8d1a68f36417-kube-api-access-4wlnk\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.273173 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.273106    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/036e577b-422c-4d64-aad7-8d1a68f36417-maas-api-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.273173 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.273121    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-maas-api-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.275553 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.275524    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n" pod "81eda2bb-32a2-48dd-b3d0-9194f3170c1d" (UID: "81eda2bb-32a2-48dd-b3d0-9194f3170c1d"). InnerVolumeSpecName "kube-api-access-btz7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:19.314529 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.314506    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:19.341685 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.320156    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:19.377461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.376540    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca\") pod \"e10fad34-9218-48f1-863f-05b54184b28f\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") "
Nov 04 01:30:19.378090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.378049    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv\" (UniqueName: \"kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv\") pod \"e10fad34-9218-48f1-863f-05b54184b28f\" (UID: \"e10fad34-9218-48f1-863f-05b54184b28f\") "
Nov 04 01:30:19.379447 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.379420    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btz7n\" (UniqueName: \"kubernetes.io/projected/81eda2bb-32a2-48dd-b3d0-9194f3170c1d-kube-api-access-btz7n\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.383124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.382916    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv" pod "e10fad34-9218-48f1-863f-05b54184b28f" (UID: "e10fad34-9218-48f1-863f-05b54184b28f"). InnerVolumeSpecName "kube-api-access-7r7dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:19.383680 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.383353    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca" pod "e10fad34-9218-48f1-863f-05b54184b28f" (UID: "e10fad34-9218-48f1-863f-05b54184b28f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:19.484516 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.480424    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r7dv\" (UniqueName: \"kubernetes.io/projected/e10fad34-9218-48f1-863f-05b54184b28f-kube-api-access-7r7dv\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.484516 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.480462    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e10fad34-9218-48f1-863f-05b54184b28f-maas-api-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:19.554195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.552781    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:19.763883 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.763809    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"
Nov 04 01:30:19.764038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.763819    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz" event={"ID":"e10fad34-9218-48f1-863f-05b54184b28f","Type":"ContainerDied","Data":"2a085ea482596a53225eb7f5a217fbd6243f81a9c10fe40bf7680251d0a5ad95"}
Nov 04 01:30:19.764038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.763985    4434 scope.go:122] "RemoveContainer" containerID="778b2788dc1069ec8881f5b5aed2fd5869188ec8c6857d4ed0deeb4b44ab71f3"
Nov 04 01:30:19.765306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.765277    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj" event={"ID":"036e577b-422c-4d64-aad7-8d1a68f36417","Type":"ContainerDied","Data":"c71deb9da13a0d0ec7187df0945211f5dac94d24b431ab4dbebede7a489aafef"}
Nov 04 01:30:19.765306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.765300    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"
Nov 04 01:30:19.767038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.767014    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"
Nov 04 01:30:19.767156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.767062    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt" event={"ID":"81eda2bb-32a2-48dd-b3d0-9194f3170c1d","Type":"ContainerDied","Data":"fa5c120cbef44d73614d3e5186c904abd983df1507ff525a9c05865a07f1ba45"}
Nov 04 01:30:19.773833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.773815    4434 scope.go:122] "RemoveContainer" containerID="5e7e519e33724f27282f056062a25b721763045d86d981f905094d5b29edaa2f"
Nov 04 01:30:19.781926 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.781908    4434 scope.go:122] "RemoveContainer" containerID="ad13872ae06993f6205f790094174e312979d0717bbef063e6f5e138039fe980"
Nov 04 01:30:19.942432 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.942398    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"]
Nov 04 01:30:19.954563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:19.954539    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-3dc6c8-29895930-5jppt"]
Nov 04 01:30:20.068988 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:20.068953    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"]
Nov 04 01:30:20.078118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:20.078087    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w6-9a0432-29895930-cjgvz"]
Nov 04 01:30:20.192049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:20.192017    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"]
Nov 04 01:30:20.196981 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:20.196952    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e65b23-29895930-r76vj"]
Nov 04 01:30:20.433286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:20.433254    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:21.477508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:21.477463    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" path="/var/lib/kubelet/pods/036e577b-422c-4d64-aad7-8d1a68f36417/volumes"
Nov 04 01:30:21.478120 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:21.478098    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" path="/var/lib/kubelet/pods/81eda2bb-32a2-48dd-b3d0-9194f3170c1d/volumes"
Nov 04 01:30:21.478558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:21.478538    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e10fad34-9218-48f1-863f-05b54184b28f" path="/var/lib/kubelet/pods/e10fad34-9218-48f1-863f-05b54184b28f/volumes"
Nov 04 01:30:21.898638 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:21.898603    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:30:23.491462 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:23.491433    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:23.708390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:23.708356    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:23.959374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:23.959342    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:26.274967 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:26.274917    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:26.823018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:26.822975    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:27.049707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:27.049637    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:32.015279 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:32.015238    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:34.786647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:34.786615    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:37.689377 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:37.689338    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:40.961488 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:40.961449    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:43.115033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:43.114999    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:47.504829 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:47.504790    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"]
Nov 04 01:30:47.594416 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:47.594381    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-serving-cert: secret "maas-api-e2e-shared-a-w6-9a0432-serving-cert" not found
Nov 04 01:30:47.594603 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:47.594473    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls podName:b9e972b7-1b70-4122-a7d5-c45e327d6281 nodeName:}" failed. No retries permitted until 2026-11-04 01:30:48.094453357 +0000 UTC m=+2016.756593410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls") pod "maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281") : secret "maas-api-e2e-shared-a-w6-9a0432-serving-cert" not found
Nov 04 01:30:47.756821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:47.756710    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:47.953072 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:47.953035    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"]
Nov 04 01:30:47.954867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:47.954836    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" podUID="b9e972b7-1b70-4122-a7d5-c45e327d6281" containerName="maas-api" containerID="cri-o://9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344" gracePeriod=30
Nov 04 01:30:48.103345 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:48.103120    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-serving-cert: secret "maas-api-e2e-shared-a-w6-9a0432-serving-cert" not found
Nov 04 01:30:48.103345 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:48.103202    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls podName:b9e972b7-1b70-4122-a7d5-c45e327d6281 nodeName:}" failed. No retries permitted until 2026-11-04 01:30:49.103182287 +0000 UTC m=+2017.765322343 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls") pod "maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281") : secret "maas-api-e2e-shared-a-w6-9a0432-serving-cert" not found
Nov 04 01:30:48.270719 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.270675    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:48.289869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.289509    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:30:48.408313 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.408276    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs\") pod \"b9e972b7-1b70-4122-a7d5-c45e327d6281\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") "
Nov 04 01:30:48.408507 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.408360    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9\" (UniqueName: \"kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9\") pod \"b9e972b7-1b70-4122-a7d5-c45e327d6281\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") "
Nov 04 01:30:48.408507 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.408458    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca\") pod \"b9e972b7-1b70-4122-a7d5-c45e327d6281\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") "
Nov 04 01:30:48.408638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.408561    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\") pod \"b9e972b7-1b70-4122-a7d5-c45e327d6281\" (UID: \"b9e972b7-1b70-4122-a7d5-c45e327d6281\") "
Nov 04 01:30:48.408919 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.408883    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca" pod "b9e972b7-1b70-4122-a7d5-c45e327d6281" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:48.409277 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.409233    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b9e972b7-1b70-4122-a7d5-c45e327d6281-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:48.410577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.410549    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs" pod "b9e972b7-1b70-4122-a7d5-c45e327d6281" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:30:48.410577 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.410565    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls" pod "b9e972b7-1b70-4122-a7d5-c45e327d6281" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:30:48.410769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.410626    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9" pod "b9e972b7-1b70-4122-a7d5-c45e327d6281" (UID: "b9e972b7-1b70-4122-a7d5-c45e327d6281"). InnerVolumeSpecName "kube-api-access-rc5t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:48.482170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.482134    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:48.510797 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.510759    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:48.510797 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.510791    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc5t9\" (UniqueName: \"kubernetes.io/projected/b9e972b7-1b70-4122-a7d5-c45e327d6281-kube-api-access-rc5t9\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:48.510797 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.510803    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b9e972b7-1b70-4122-a7d5-c45e327d6281-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:48.962406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.962374    4434 generic.go:356] "Generic (PLEG): container finished" podID="b9e972b7-1b70-4122-a7d5-c45e327d6281" containerID="9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344" exitCode=0
Nov 04 01:30:48.962586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.962424    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"
Nov 04 01:30:48.962586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.962451    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" event={"ID":"b9e972b7-1b70-4122-a7d5-c45e327d6281","Type":"ContainerDied","Data":"9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344"}
Nov 04 01:30:48.962586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.962507    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c" event={"ID":"b9e972b7-1b70-4122-a7d5-c45e327d6281","Type":"ContainerDied","Data":"5c2d608ae693470df3fcf74a7cd3fbe1ad0e1cb2c67b48fd5f67d6634a25bf11"}
Nov 04 01:30:48.962586 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.962529    4434 scope.go:122] "RemoveContainer" containerID="9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344"
Nov 04 01:30:48.972834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.972816    4434 scope.go:122] "RemoveContainer" containerID="9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344"
Nov 04 01:30:48.973124 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:48.973096    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344\": container with ID starting with 9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344 not found: ID does not exist" containerID="9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344"
Nov 04 01:30:48.973193 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:48.973136    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344"} err="failed to get container status \"9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344\": rpc error: code = NotFound desc = could not find container \"9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344\": container with ID starting with 9623f77e5775c803ad692a4068dceef122249c4e9503c59df594efa3a34de344 not found: ID does not exist"
Nov 04 01:30:49.136090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:49.136054    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"]
Nov 04 01:30:49.153096 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:49.153064    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-9a0432-9f75c4ff7-qkz8c"]
Nov 04 01:30:49.422194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:49.422154    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:49.494259 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:49.494222    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9e972b7-1b70-4122-a7d5-c45e327d6281" path="/var/lib/kubelet/pods/b9e972b7-1b70-4122-a7d5-c45e327d6281/volumes"
Nov 04 01:30:49.679217 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:49.679141    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:49.893487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:49.893452    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:50.555873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:50.555844    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:51.898385 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:51.898347    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:30:52.953934 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:52.953896    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd" podUID="24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" containerName="payload-pre-processing" containerID="cri-o://035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e" gracePeriod=25
Nov 04 01:30:53.228217 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.228193    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:30:53.274918 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.274887    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p\" (UniqueName: \"kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p\") pod \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") "
Nov 04 01:30:53.275085 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.274944    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume\") pod \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\" (UID: \"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a\") "
Nov 04 01:30:53.275418 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.275386    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume" pod "24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" (UID: "24ff5a75-4172-4bf1-8826-97ff4ec4eb2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:53.275752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.275730    4434 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-plugins-config-volume\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:53.277132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.277110    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p" pod "24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" (UID: "24ff5a75-4172-4bf1-8826-97ff4ec4eb2a"). InnerVolumeSpecName "kube-api-access-wc58p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:53.377584 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.377544    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc58p\" (UniqueName: \"kubernetes.io/projected/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a-kube-api-access-wc58p\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:53.992118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.992076    4434 generic.go:356] "Generic (PLEG): container finished" podID="24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" containerID="035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e" exitCode=0
Nov 04 01:30:53.992587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.992144    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"
Nov 04 01:30:53.992587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.992163    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd" event={"ID":"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a","Type":"ContainerDied","Data":"035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e"}
Nov 04 01:30:53.992587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.992227    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd" event={"ID":"24ff5a75-4172-4bf1-8826-97ff4ec4eb2a","Type":"ContainerDied","Data":"5cccb59a8e8b5d106b160c39c894faf477096643200a4ea5000bf918234a5b01"}
Nov 04 01:30:53.992587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:53.992248    4434 scope.go:122] "RemoveContainer" containerID="035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e"
Nov 04 01:30:54.001727 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.001709    4434 scope.go:122] "RemoveContainer" containerID="035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e"
Nov 04 01:30:54.002014 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:54.001994    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e\": container with ID starting with 035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e not found: ID does not exist" containerID="035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e"
Nov 04 01:30:54.002065 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.002024    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e"} err="failed to get container status \"035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e\": rpc error: code = NotFound desc = could not find container \"035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e\": container with ID starting with 035a8646ca1786810617d2c32f562c516031eff558336055775c7a4ec259477e not found: ID does not exist"
Nov 04 01:30:54.174488 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.174449    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"]
Nov 04 01:30:54.180997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.180969    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5186fe-6544c5f8c5-ld4zd"]
Nov 04 01:30:54.483067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.483034    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"]
Nov 04 01:30:54.483288 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.483264    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs" podUID="ae109d0f-8ced-48dd-892b-25fb80868747" containerName="maas-api" containerID="cri-o://3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d" gracePeriod=30
Nov 04 01:30:54.657471 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.657400    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:54.820462 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.820431    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:30:54.899846 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.899801    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22\" (UniqueName: \"kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22\") pod \"ae109d0f-8ced-48dd-892b-25fb80868747\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") "
Nov 04 01:30:54.900039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.899860    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs\") pod \"ae109d0f-8ced-48dd-892b-25fb80868747\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") "
Nov 04 01:30:54.900039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.899981    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls\") pod \"ae109d0f-8ced-48dd-892b-25fb80868747\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") "
Nov 04 01:30:54.900171 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.900093    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca\") pod \"ae109d0f-8ced-48dd-892b-25fb80868747\" (UID: \"ae109d0f-8ced-48dd-892b-25fb80868747\") "
Nov 04 01:30:54.902517 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.902482    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca" pod "ae109d0f-8ced-48dd-892b-25fb80868747" (UID: "ae109d0f-8ced-48dd-892b-25fb80868747"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:30:54.905735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.905657    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs" pod "ae109d0f-8ced-48dd-892b-25fb80868747" (UID: "ae109d0f-8ced-48dd-892b-25fb80868747"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:30:54.906043 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.905917    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22" pod "ae109d0f-8ced-48dd-892b-25fb80868747" (UID: "ae109d0f-8ced-48dd-892b-25fb80868747"). InnerVolumeSpecName "kube-api-access-c6p22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:30:54.906043 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:54.905997    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls" pod "ae109d0f-8ced-48dd-892b-25fb80868747" (UID: "ae109d0f-8ced-48dd-892b-25fb80868747"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:30:55.000509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.000470    4434 generic.go:356] "Generic (PLEG): container finished" podID="ae109d0f-8ced-48dd-892b-25fb80868747" containerID="3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d" exitCode=0
Nov 04 01:30:55.001053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.000516    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"
Nov 04 01:30:55.001053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.000546    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs" event={"ID":"ae109d0f-8ced-48dd-892b-25fb80868747","Type":"ContainerDied","Data":"3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d"}
Nov 04 01:30:55.001053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.000612    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs" event={"ID":"ae109d0f-8ced-48dd-892b-25fb80868747","Type":"ContainerDied","Data":"c76a9266da2777e7fda740c9885925bfd4d422c44eab170dbe1eca348856d024"}
Nov 04 01:30:55.001053 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.000645    4434 scope.go:122] "RemoveContainer" containerID="3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d"
Nov 04 01:30:55.001617 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.001592    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:55.001780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.001621    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae109d0f-8ced-48dd-892b-25fb80868747-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:55.001780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.001635    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6p22\" (UniqueName: \"kubernetes.io/projected/ae109d0f-8ced-48dd-892b-25fb80868747-kube-api-access-c6p22\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:55.001780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.001647    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae109d0f-8ced-48dd-892b-25fb80868747-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:30:55.012382 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.012359    4434 scope.go:122] "RemoveContainer" containerID="3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d"
Nov 04 01:30:55.012696 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:30:55.012675    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d\": container with ID starting with 3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d not found: ID does not exist" containerID="3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d"
Nov 04 01:30:55.012790 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.012707    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d"} err="failed to get container status \"3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d\": rpc error: code = NotFound desc = could not find container \"3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d\": container with ID starting with 3c18c8b6206a5ae108635e7d0a1c787ce36ddd07dafeb3918b67c91713d81a1d not found: ID does not exist"
Nov 04 01:30:55.171330 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.171291    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"]
Nov 04 01:30:55.179954 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.179925    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-e65b23-677c484f7d-58hrs"]
Nov 04 01:30:55.475167 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.475092    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" path="/var/lib/kubelet/pods/24ff5a75-4172-4bf1-8826-97ff4ec4eb2a/volumes"
Nov 04 01:30:55.475547 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:55.475534    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae109d0f-8ced-48dd-892b-25fb80868747" path="/var/lib/kubelet/pods/ae109d0f-8ced-48dd-892b-25fb80868747/volumes"
Nov 04 01:30:56.811828 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:56.811796    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:57.075801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:57.075724    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:57.565212 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:57.565174    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:58.299338 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:58.299296    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:58.545543 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:58.545509    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:30:58.785428 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:30:58.785392    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:01.543086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:01.543054    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"]
Nov 04 01:31:01.543569 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:01.543250    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" podUID="623be538-08d1-4ad5-806c-c7b5383d9a32" containerName="maas-api" containerID="cri-o://9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74" gracePeriod=30
Nov 04 01:31:01.695948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:01.695876    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:01.866870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:01.866841    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:31:02.014969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.013417    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca\") pod \"623be538-08d1-4ad5-806c-c7b5383d9a32\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") "
Nov 04 01:31:02.014969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.013517    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs\") pod \"623be538-08d1-4ad5-806c-c7b5383d9a32\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") "
Nov 04 01:31:02.014969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.013646    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd\" (UniqueName: \"kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd\") pod \"623be538-08d1-4ad5-806c-c7b5383d9a32\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") "
Nov 04 01:31:02.014969 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.013709    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\") pod \"623be538-08d1-4ad5-806c-c7b5383d9a32\" (UID: \"623be538-08d1-4ad5-806c-c7b5383d9a32\") "
Nov 04 01:31:02.018881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.017860    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls" pod "623be538-08d1-4ad5-806c-c7b5383d9a32" (UID: "623be538-08d1-4ad5-806c-c7b5383d9a32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:31:02.019490 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.019277    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca" pod "623be538-08d1-4ad5-806c-c7b5383d9a32" (UID: "623be538-08d1-4ad5-806c-c7b5383d9a32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:31:02.019775 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.019743    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs" pod "623be538-08d1-4ad5-806c-c7b5383d9a32" (UID: "623be538-08d1-4ad5-806c-c7b5383d9a32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:31:02.022251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.022220    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd" pod "623be538-08d1-4ad5-806c-c7b5383d9a32" (UID: "623be538-08d1-4ad5-806c-c7b5383d9a32"). InnerVolumeSpecName "kube-api-access-49rkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:02.044570 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.044538    4434 generic.go:356] "Generic (PLEG): container finished" podID="623be538-08d1-4ad5-806c-c7b5383d9a32" containerID="9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74" exitCode=0
Nov 04 01:31:02.044784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.044614    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"
Nov 04 01:31:02.044784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.044719    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" event={"ID":"623be538-08d1-4ad5-806c-c7b5383d9a32","Type":"ContainerDied","Data":"9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74"}
Nov 04 01:31:02.044784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.044756    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp" event={"ID":"623be538-08d1-4ad5-806c-c7b5383d9a32","Type":"ContainerDied","Data":"dc14ee58bcc37efa7f16869ecee053d68a5bf925c3ec748df5393dcf8cfe60c1"}
Nov 04 01:31:02.044784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.044782    4434 scope.go:122] "RemoveContainer" containerID="9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74"
Nov 04 01:31:02.054750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.054725    4434 scope.go:122] "RemoveContainer" containerID="9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74"
Nov 04 01:31:02.055069 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:02.055042    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74\": container with ID starting with 9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74 not found: ID does not exist" containerID="9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74"
Nov 04 01:31:02.055153 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.055083    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74"} err="failed to get container status \"9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74\": rpc error: code = NotFound desc = could not find container \"9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74\": container with ID starting with 9c72daff7c2daaec1ad5c0fa2d820c8f695289ca27b8dfdb60d50751e8d4da74 not found: ID does not exist"
Nov 04 01:31:02.122720 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.122614    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/623be538-08d1-4ad5-806c-c7b5383d9a32-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:02.122720 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.122699    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:02.122720 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.122719    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49rkd\" (UniqueName: \"kubernetes.io/projected/623be538-08d1-4ad5-806c-c7b5383d9a32-kube-api-access-49rkd\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:02.122996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.122732    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/623be538-08d1-4ad5-806c-c7b5383d9a32-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:02.213062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.213022    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"]
Nov 04 01:31:02.215764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:02.215738    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5186fe-6f46954d4f-vqlwp"]
Nov 04 01:31:03.408388 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:03.408353    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:03.475855 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:03.475819    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="623be538-08d1-4ad5-806c-c7b5383d9a32" path="/var/lib/kubelet/pods/623be538-08d1-4ad5-806c-c7b5383d9a32/volumes"
Nov 04 01:31:03.648402 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:03.648368    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:03.846716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:03.846611    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:04.135008 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.134972    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"]
Nov 04 01:31:04.135252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.135226    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy" containerID="cri-o://ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc" gracePeriod=30
Nov 04 01:31:04.148524 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.148486    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"]
Nov 04 01:31:04.148774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.148739    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy" containerID="cri-o://9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52" gracePeriod=30
Nov 04 01:31:04.160892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.160861    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:04.163374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.163350    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"]
Nov 04 01:31:04.163607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.163567    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy" containerID="cri-o://d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae" gracePeriod=30
Nov 04 01:31:04.430860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:04.430773    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:05.973886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:05.973854    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:06.423483 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:06.423452    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:07.167856 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:07.167810    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:07.479656 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:07.479575    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:07.781640 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:07.781553    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:09.463618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.463290    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:31:09.484508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.484418    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:31:09.507908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.507856    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:09.514784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.514765    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:31:09.563170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563141    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.563321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563187    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.563321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563240    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.563321 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563269    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.563484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563335    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.563484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563364    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.563484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563420    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.563484 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563463    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563493    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563522    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563555    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563557    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563600    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563631    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563646    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.563715 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563692    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563706    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563774    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563865    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563880    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563909    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563957    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.563995    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564031    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564063    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.564136 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564120    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.564700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564218    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket\") pod \"d49dd45c-5812-43e4-a378-efb30ce97358\" (UID: \"d49dd45c-5812-43e4-a378-efb30ce97358\") "
Nov 04 01:31:09.564700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564284    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.564700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564328    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.564700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564375    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket\") pod \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\" (UID: \"6a73eb28-2cfd-4566-b73e-fedc8d771a1f\") "
Nov 04 01:31:09.564700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564405    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket\") pod \"6c1ce881-d58b-49e8-a18e-58b36f34d299\" (UID: \"6c1ce881-d58b-49e8-a18e-58b36f34d299\") "
Nov 04 01:31:09.564997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564894    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.565451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565119    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.565451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.564839    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.565451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565166    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.565451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565219    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:31:09.565451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565301    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.565952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565499    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:31:09.565952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565591    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:31:09.565952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565747    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566232    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566258    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566274    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566289    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566307    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566321    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.566845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566762    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c1ce881-d58b-49e8-a18e-58b36f34d299-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.567092 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566916    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d49dd45c-5812-43e4-a378-efb30ce97358-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.567092 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566938    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.567852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566953    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.567852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.567837    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.567852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.565943    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.568080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.566038    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.568080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.567330    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.568301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.568278    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "kube-api-access-5psvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:09.568440 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.568413    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "kube-api-access-nwsg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:09.568616 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.568580    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:09.568770 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.568746    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.569041 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.568829    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.569041 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.568983    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:31:09.570073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.570047    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token" pod "6a73eb28-2cfd-4566-b73e-fedc8d771a1f" (UID: "6a73eb28-2cfd-4566-b73e-fedc8d771a1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:09.570205 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.570145    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:31:09.570316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.570296    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:31:09.570374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.570321    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:09.570412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.570373    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7" pod "6c1ce881-d58b-49e8-a18e-58b36f34d299" (UID: "6c1ce881-d58b-49e8-a18e-58b36f34d299"). InnerVolumeSpecName "kube-api-access-fxjr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:09.571764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.571738    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token" pod "d49dd45c-5812-43e4-a378-efb30ce97358" (UID: "d49dd45c-5812-43e4-a378-efb30ce97358"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668653    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668694    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668705    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668715    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d49dd45c-5812-43e4-a378-efb30ce97358-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668725    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668733    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.668742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668741    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5psvz\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-kube-api-access-5psvz\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668752    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d49dd45c-5812-43e4-a378-efb30ce97358-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668760    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c1ce881-d58b-49e8-a18e-58b36f34d299-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668768    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwsg9\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-kube-api-access-nwsg9\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668778    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668786    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c1ce881-d58b-49e8-a18e-58b36f34d299-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668793    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d49dd45c-5812-43e4-a378-efb30ce97358-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668802    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a73eb28-2cfd-4566-b73e-fedc8d771a1f-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:09.669075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:09.668811    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxjr7\" (UniqueName: \"kubernetes.io/projected/6c1ce881-d58b-49e8-a18e-58b36f34d299-kube-api-access-fxjr7\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:10.104019 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.103937    4434 generic.go:356] "Generic (PLEG): container finished" podID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerID="9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52" exitCode=0
Nov 04 01:31:10.104019 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.103986    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"
Nov 04 01:31:10.104240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.104019    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" event={"ID":"6a73eb28-2cfd-4566-b73e-fedc8d771a1f","Type":"ContainerDied","Data":"9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52"}
Nov 04 01:31:10.104240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.104075    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr" event={"ID":"6a73eb28-2cfd-4566-b73e-fedc8d771a1f","Type":"ContainerDied","Data":"719edbc4678035a3da7fa448c23642ca5129aab39f956ba850c617571423b259"}
Nov 04 01:31:10.104240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.104103    4434 scope.go:122] "RemoveContainer" containerID="9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52"
Nov 04 01:31:10.105491 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.105466    4434 generic.go:356] "Generic (PLEG): container finished" podID="d49dd45c-5812-43e4-a378-efb30ce97358" containerID="d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae" exitCode=0
Nov 04 01:31:10.105638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.105489    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" event={"ID":"d49dd45c-5812-43e4-a378-efb30ce97358","Type":"ContainerDied","Data":"d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae"}
Nov 04 01:31:10.105638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.105509    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"
Nov 04 01:31:10.105638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.105537    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv" event={"ID":"d49dd45c-5812-43e4-a378-efb30ce97358","Type":"ContainerDied","Data":"ae1dc246f02de6ea3b52d9e847508bc9db3bad20e56bf02beb87d492276b158a"}
Nov 04 01:31:10.107170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.107133    4434 generic.go:356] "Generic (PLEG): container finished" podID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerID="ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc" exitCode=0
Nov 04 01:31:10.107274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.107183    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"
Nov 04 01:31:10.107274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.107196    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" event={"ID":"6c1ce881-d58b-49e8-a18e-58b36f34d299","Type":"ContainerDied","Data":"ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc"}
Nov 04 01:31:10.107274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.107239    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w" event={"ID":"6c1ce881-d58b-49e8-a18e-58b36f34d299","Type":"ContainerDied","Data":"d0e166690303b8d52057322205193a7e51aa85d937e4adcf62a434506052f3b5"}
Nov 04 01:31:10.115374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.115356    4434 scope.go:122] "RemoveContainer" containerID="9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52"
Nov 04 01:31:10.115735 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:10.115712    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52\": container with ID starting with 9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52 not found: ID does not exist" containerID="9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52"
Nov 04 01:31:10.115854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.115745    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52"} err="failed to get container status \"9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52\": rpc error: code = NotFound desc = could not find container \"9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52\": container with ID starting with 9bfd3a5db22047f0b5c1401a05d4f88f1645e6f30f3da5b2a75565c403174a52 not found: ID does not exist"
Nov 04 01:31:10.115854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.115763    4434 scope.go:122] "RemoveContainer" containerID="d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae"
Nov 04 01:31:10.125123 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.125108    4434 scope.go:122] "RemoveContainer" containerID="d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae"
Nov 04 01:31:10.125369 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:10.125352    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae\": container with ID starting with d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae not found: ID does not exist" containerID="d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae"
Nov 04 01:31:10.125415 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.125379    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae"} err="failed to get container status \"d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae\": rpc error: code = NotFound desc = could not find container \"d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae\": container with ID starting with d3452cca4b585a532c2cda107cd1a9a54eb7eff72ff896bd05ec34006f7776ae not found: ID does not exist"
Nov 04 01:31:10.125415 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.125398    4434 scope.go:122] "RemoveContainer" containerID="ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc"
Nov 04 01:31:10.134040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.134025    4434 scope.go:122] "RemoveContainer" containerID="ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc"
Nov 04 01:31:10.134288 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:10.134271    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc\": container with ID starting with ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc not found: ID does not exist" containerID="ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc"
Nov 04 01:31:10.134341 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.134294    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc"} err="failed to get container status \"ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc\": rpc error: code = NotFound desc = could not find container \"ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc\": container with ID starting with ac714d981f04a7803d2dd517abdceb267f6246cc854a0e1a14735d9e29e9a1dc not found: ID does not exist"
Nov 04 01:31:10.280599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.280567    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"]
Nov 04 01:31:10.330844 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.330801    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-x6tfr"]
Nov 04 01:31:10.449868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.449837    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"]
Nov 04 01:31:10.462854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.462825    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-c6kcv"]
Nov 04 01:31:10.591191 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.591152    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"]
Nov 04 01:31:10.599080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:10.599054    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e65b23-openshift-default-6d67df5759-h7j8w"]
Nov 04 01:31:11.475272 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:11.475239    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" path="/var/lib/kubelet/pods/6a73eb28-2cfd-4566-b73e-fedc8d771a1f/volumes"
Nov 04 01:31:11.475754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:11.475740    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" path="/var/lib/kubelet/pods/6c1ce881-d58b-49e8-a18e-58b36f34d299/volumes"
Nov 04 01:31:11.476202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:11.476188    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" path="/var/lib/kubelet/pods/d49dd45c-5812-43e4-a378-efb30ce97358/volumes"
Nov 04 01:31:11.640819 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:11.640787    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:12.076124 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:12.075961    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:12.679788 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:12.679752    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:12.935097 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:12.935012    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:14.306867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:14.306836    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:14.511540 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:14.511502    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:14.716283 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:14.716255    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:15.405696 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:15.405648    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:16.526255 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:16.526166    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:16.733512 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:16.733463    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:20.751996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:20.751963    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:20.964011 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:20.963976    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:21.145850 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:21.145820    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:21.896958 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:21.896923    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:31:24.129762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:24.129724    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"]
Nov 04 01:31:24.130203 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:24.129972    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy" containerID="cri-o://dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3" gracePeriod=30
Nov 04 01:31:25.999009 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:25.998976    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:28.246658 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:28.246618    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:28.755689 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:28.755627    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:29.460112 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.460081    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:31:29.574024 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.573974    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574040    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574099    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574125    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574160    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574246    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574296    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574383    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574416    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:29.574446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574429    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo\") pod \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\" (UID: \"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930\") "
Nov 04 01:31:29.574742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574471    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:31:29.574742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574612    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:29.574742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574707    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:29.574901 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.574776    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:29.575103 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.575081    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.575269 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.575253    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.575389 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.575369    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.575455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.575392    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.575455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.575406    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.576733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.576700    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "kube-api-access-8s867". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:29.576733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.576701    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:31:29.576915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.576893    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:31:29.577105 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.577083    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token" pod "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" (UID: "f17bf0ce-3d87-457d-ac9c-b27d5ddcb930"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:31:29.676316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.676283    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.676316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.676311    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s867\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-kube-api-access-8s867\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.676316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.676322    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:29.676538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:29.676330    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:31:30.244392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.244356    4434 generic.go:356] "Generic (PLEG): container finished" podID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerID="dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3" exitCode=0
Nov 04 01:31:30.244571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.244402    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"
Nov 04 01:31:30.244571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.244443    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" event={"ID":"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930","Type":"ContainerDied","Data":"dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3"}
Nov 04 01:31:30.244571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.244499    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb" event={"ID":"f17bf0ce-3d87-457d-ac9c-b27d5ddcb930","Type":"ContainerDied","Data":"115cea4c8b3e443dfe429e7024a8722f123c32e6d98c8c0ee4f133dad1eac0be"}
Nov 04 01:31:30.244571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.244523    4434 scope.go:122] "RemoveContainer" containerID="dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3"
Nov 04 01:31:30.254854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.254687    4434 scope.go:122] "RemoveContainer" containerID="dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3"
Nov 04 01:31:30.254993 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:30.254972    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3\": container with ID starting with dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3 not found: ID does not exist" containerID="dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3"
Nov 04 01:31:30.255059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.255005    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3"} err="failed to get container status \"dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3\": rpc error: code = NotFound desc = could not find container \"dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3\": container with ID starting with dd893f5d04496dbeeccf077a4d3c81c4c95276b490e843770fca67d4040b6cf3 not found: ID does not exist"
Nov 04 01:31:30.471519 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.471481    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"]
Nov 04 01:31:30.479778 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:30.479756    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-nj4xb"]
Nov 04 01:31:31.132547 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:31.132497    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:31.474987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:31.474902    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" path="/var/lib/kubelet/pods/f17bf0ce-3d87-457d-ac9c-b27d5ddcb930/volumes"
Nov 04 01:31:36.281439 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:36.281393    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:41.219065 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:41.219030    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:45.565093 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:45.565063    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:47.421590 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:47.421557    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:47.881686 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:47.881629    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:48.165922 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:48.165822    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:48.882678 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:48.882632    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:51.898062 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:51.898022    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:31:52.865302 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:52.865267    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:55.403365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:55.403333    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:55.658539 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:55.658448    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:55.896581 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:55.896533    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:56.291954 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:56.291926    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:57.614599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.614564    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"]
Nov 04 01:31:57.615078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615058    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615071    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615088    4434 cpu_manager.go:414] "containerMap: removing container" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615093    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615102    4434 cpu_manager.go:414] "containerMap: removing container" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615107    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615114    4434 cpu_manager.go:414] "containerMap: removing container" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615119    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615126    4434 cpu_manager.go:414] "containerMap: removing container" podUID="ae109d0f-8ced-48dd-892b-25fb80868747" containerName="maas-api"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615131    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae109d0f-8ced-48dd-892b-25fb80868747" containerName="maas-api"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615138    4434 cpu_manager.go:414] "containerMap: removing container" podUID="24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" containerName="payload-pre-processing"
Nov 04 01:31:57.615150 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615143    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" containerName="payload-pre-processing"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615155    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615160    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615171    4434 cpu_manager.go:414] "containerMap: removing container" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615176    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615185    4434 cpu_manager.go:414] "containerMap: removing container" podUID="623be538-08d1-4ad5-806c-c7b5383d9a32" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615189    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="623be538-08d1-4ad5-806c-c7b5383d9a32" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615200    4434 cpu_manager.go:414] "containerMap: removing container" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615204    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615211    4434 cpu_manager.go:414] "containerMap: removing container" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615217    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615224    4434 cpu_manager.go:414] "containerMap: removing container" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615229    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615236    4434 cpu_manager.go:414] "containerMap: removing container" podUID="b9e972b7-1b70-4122-a7d5-c45e327d6281" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615240    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9e972b7-1b70-4122-a7d5-c45e327d6281" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615251    4434 cpu_manager.go:414] "containerMap: removing container" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615255    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615322    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615331    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615338    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a73eb28-2cfd-4566-b73e-fedc8d771a1f" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615344    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615349    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615354    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae109d0f-8ced-48dd-892b-25fb80868747" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615361    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="24ff5a75-4172-4bf1-8826-97ff4ec4eb2a" containerName="payload-pre-processing"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615367    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615373    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615379    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="623be538-08d1-4ad5-806c-c7b5383d9a32" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615384    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c1ce881-d58b-49e8-a18e-58b36f34d299" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615388    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="f17bf0ce-3d87-457d-ac9c-b27d5ddcb930" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615393    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="d49dd45c-5812-43e4-a378-efb30ce97358" containerName="istio-proxy"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615397    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9e972b7-1b70-4122-a7d5-c45e327d6281" containerName="maas-api"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615402    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:31:57.615487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.615406    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:31:57.618781 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.618763    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.672783 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.672755    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-843be0-serving-cert\""
Nov 04 01:31:57.712032 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.712007    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.712174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.712051    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.712174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.712079    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x96sd\" (UniqueName: \"kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.712259 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.712234    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.725481 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.725455    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"]
Nov 04 01:31:57.813642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.813609    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.813824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.813659    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.813824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.813710    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x96sd\" (UniqueName: \"kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.813824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.813746    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.813956 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:57.813859    4434 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-serving-cert: secret "maas-api-e2e-shared-b-w2-843be0-serving-cert" not found
Nov 04 01:31:57.813956 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:31:57.813931    4434 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls podName:a922c3b3-2d94-438d-b0a0-3b15c97d0d02 nodeName:}" failed. No retries permitted until 2026-11-04 01:31:58.313912625 +0000 UTC m=+2086.976052674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls") pod "maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" (UID: "a922c3b3-2d94-438d-b0a0-3b15c97d0d02") : secret "maas-api-e2e-shared-b-w2-843be0-serving-cert" not found
Nov 04 01:31:57.814470 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.814449    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.816156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.816132    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.879452 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.879414    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x96sd\" (UniqueName: \"kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:57.931520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:57.931464    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:58.147272 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:58.147181    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:31:58.324200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:58.324164    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:58.326537 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:58.326518    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:58.531352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:58.531268    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:31:58.910295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:58.910261    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"]
Nov 04 01:31:58.914510 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:31:58.914482    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda922c3b3_2d94_438d_b0a0_3b15c97d0d02.slice/crio-41ca26e005a07aa20c5ea31d216202da7df89059801e9e3f579840008b2b25c9 WatchSource:0}: Error finding container 41ca26e005a07aa20c5ea31d216202da7df89059801e9e3f579840008b2b25c9: Status 404 returned error can't find the container with id 41ca26e005a07aa20c5ea31d216202da7df89059801e9e3f579840008b2b25c9
Nov 04 01:31:58.915918 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:58.915901    4434 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:31:59.412814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:31:59.412779    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" event={"ID":"a922c3b3-2d94-438d-b0a0-3b15c97d0d02","Type":"ContainerStarted","Data":"41ca26e005a07aa20c5ea31d216202da7df89059801e9e3f579840008b2b25c9"}
Nov 04 01:32:00.419897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:00.419863    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" event={"ID":"a922c3b3-2d94-438d-b0a0-3b15c97d0d02","Type":"ContainerStarted","Data":"dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3"}
Nov 04 01:32:00.420310 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:00.420010    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:32:00.538986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:00.538935    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" podStartSLOduration=2.597736481 podStartE2EDuration="3.538920278s" totalImagesPullingTime="941.183797ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:31:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:32:00.537461448 +0000 UTC m=+2089.199601533" watchObservedRunningTime="2026-11-04 01:32:00.538920278 +0000 UTC m=+2089.201060339"
Nov 04 01:32:01.190154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:01.190117    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"]
Nov 04 01:32:01.239562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:01.239527    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"]
Nov 04 01:32:01.372357 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:01.372307    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:03.694902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:03.694863    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:06.431566 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:06.431536    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:32:07.408086 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:07.408052    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:09.342996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:09.342963    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:13.771779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:13.771744    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:15.421867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:15.421833    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:15.703903 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:15.703823    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:15.952560 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:15.952531    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:16.210451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.210418    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main" containerID="cri-o://033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128" gracePeriod=45
Nov 04 01:32:16.259549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.259493    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" containerID="cri-o://39a8834311af0a39e30239583c521a7e6d0740ed13311f08313ea13ca072caeb" gracePeriod=45
Nov 04 01:32:16.501899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.501871    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:16.509298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.509266    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:32:16.511540 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.511516    4434 generic.go:356] "Generic (PLEG): container finished" podID="9112be4e-695a-4087-a364-fd290ae66db3" containerID="39a8834311af0a39e30239583c521a7e6d0740ed13311f08313ea13ca072caeb" exitCode=0
Nov 04 01:32:16.511645 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.511594    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" event={"ID":"9112be4e-695a-4087-a364-fd290ae66db3","Type":"ContainerDied","Data":"39a8834311af0a39e30239583c521a7e6d0740ed13311f08313ea13ca072caeb"}
Nov 04 01:32:16.513055 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.513024    4434 generic.go:356] "Generic (PLEG): container finished" podID="814094e6-b400-482c-b461-3957063e72e3" containerID="033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128" exitCode=0
Nov 04 01:32:16.513174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.513061    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"
Nov 04 01:32:16.513174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.513067    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" event={"ID":"814094e6-b400-482c-b461-3957063e72e3","Type":"ContainerDied","Data":"033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128"}
Nov 04 01:32:16.513174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.513094    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz" event={"ID":"814094e6-b400-482c-b461-3957063e72e3","Type":"ContainerDied","Data":"f556dd8f54a85d5bc0050d031169efcb06d83875806f75477e3c9ff3377e210a"}
Nov 04 01:32:16.513174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.513115    4434 scope.go:122] "RemoveContainer" containerID="033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128"
Nov 04 01:32:16.522005 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.521920    4434 scope.go:122] "RemoveContainer" containerID="033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128"
Nov 04 01:32:16.522260 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:32:16.522231    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128\": container with ID starting with 033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128 not found: ID does not exist" containerID="033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128"
Nov 04 01:32:16.522335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.522272    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128"} err="failed to get container status \"033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128\": rpc error: code = NotFound desc = could not find container \"033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128\": container with ID starting with 033a3fb896f890b5d56370420bb22c373cc4d9a216fc73c6afafb7b74df99128 not found: ID does not exist"
Nov 04 01:32:16.545297 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.545280    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:32:16.555315 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555297    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45\" (UniqueName: \"kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45\") pod \"814094e6-b400-482c-b461-3957063e72e3\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") "
Nov 04 01:32:16.555411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555327    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm\") pod \"814094e6-b400-482c-b461-3957063e72e3\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") "
Nov 04 01:32:16.555411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555362    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache\") pod \"814094e6-b400-482c-b461-3957063e72e3\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") "
Nov 04 01:32:16.555545 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555464    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir\") pod \"814094e6-b400-482c-b461-3957063e72e3\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") "
Nov 04 01:32:16.555737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555575    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache" pod "814094e6-b400-482c-b461-3957063e72e3" (UID: "814094e6-b400-482c-b461-3957063e72e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.555737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555591    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs\" (UniqueName: \"kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs\") pod \"814094e6-b400-482c-b461-3957063e72e3\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") "
Nov 04 01:32:16.555737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555691    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home\") pod \"814094e6-b400-482c-b461-3957063e72e3\" (UID: \"814094e6-b400-482c-b461-3957063e72e3\") "
Nov 04 01:32:16.555737 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555720    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir" pod "814094e6-b400-482c-b461-3957063e72e3" (UID: "814094e6-b400-482c-b461-3957063e72e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.555977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.555942    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home" pod "814094e6-b400-482c-b461-3957063e72e3" (UID: "814094e6-b400-482c-b461-3957063e72e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.556343 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.556320    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.556427 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.556349    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.556427 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.556366    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.557610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.557592    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm" pod "814094e6-b400-482c-b461-3957063e72e3" (UID: "814094e6-b400-482c-b461-3957063e72e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.557709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.557639    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45" pod "814094e6-b400-482c-b461-3957063e72e3" (UID: "814094e6-b400-482c-b461-3957063e72e3"). InnerVolumeSpecName "kube-api-access-f7m45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:16.557709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.557644    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs" pod "814094e6-b400-482c-b461-3957063e72e3" (UID: "814094e6-b400-482c-b461-3957063e72e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:16.657805 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.657770    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm\") pod \"9112be4e-695a-4087-a364-fd290ae66db3\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") "
Nov 04 01:32:16.657992 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.657844    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home\") pod \"9112be4e-695a-4087-a364-fd290ae66db3\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") "
Nov 04 01:32:16.657992 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.657880    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs\" (UniqueName: \"kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs\") pod \"9112be4e-695a-4087-a364-fd290ae66db3\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") "
Nov 04 01:32:16.657992 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.657947    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache\") pod \"9112be4e-695a-4087-a364-fd290ae66db3\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") "
Nov 04 01:32:16.658180 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.657995    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir\") pod \"9112be4e-695a-4087-a364-fd290ae66db3\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") "
Nov 04 01:32:16.658180 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658109    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678\" (UniqueName: \"kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678\") pod \"9112be4e-695a-4087-a364-fd290ae66db3\" (UID: \"9112be4e-695a-4087-a364-fd290ae66db3\") "
Nov 04 01:32:16.658180 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658154    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home" pod "9112be4e-695a-4087-a364-fd290ae66db3" (UID: "9112be4e-695a-4087-a364-fd290ae66db3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.658548 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658509    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir" pod "9112be4e-695a-4087-a364-fd290ae66db3" (UID: "9112be4e-695a-4087-a364-fd290ae66db3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.658787 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658766    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.658890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658795    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7m45\" (UniqueName: \"kubernetes.io/projected/814094e6-b400-482c-b461-3957063e72e3-kube-api-access-f7m45\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.658890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658810    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814094e6-b400-482c-b461-3957063e72e3-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.658890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658823    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814094e6-b400-482c-b461-3957063e72e3-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.658890 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.658835    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.659090 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.659067    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache" pod "9112be4e-695a-4087-a364-fd290ae66db3" (UID: "9112be4e-695a-4087-a364-fd290ae66db3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.660201 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.660176    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm" pod "9112be4e-695a-4087-a364-fd290ae66db3" (UID: "9112be4e-695a-4087-a364-fd290ae66db3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:16.660391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.660364    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs" pod "9112be4e-695a-4087-a364-fd290ae66db3" (UID: "9112be4e-695a-4087-a364-fd290ae66db3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:16.660555 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.660534    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678" pod "9112be4e-695a-4087-a364-fd290ae66db3" (UID: "9112be4e-695a-4087-a364-fd290ae66db3"). InnerVolumeSpecName "kube-api-access-ss678". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:16.759951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.759877    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss678\" (UniqueName: \"kubernetes.io/projected/9112be4e-695a-4087-a364-fd290ae66db3-kube-api-access-ss678\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.759951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.759904    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.759951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.759914    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9112be4e-695a-4087-a364-fd290ae66db3-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.759951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.759924    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9112be4e-695a-4087-a364-fd290ae66db3-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:16.951065 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.951033    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"]
Nov 04 01:32:16.955410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:16.955385    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/premium-w6-678aaf-kserve-55d99bc649-p29hz"]
Nov 04 01:32:17.475858 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.475826    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="814094e6-b400-482c-b461-3957063e72e3" path="/var/lib/kubelet/pods/814094e6-b400-482c-b461-3957063e72e3/volumes"
Nov 04 01:32:17.513615 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.513570    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8000/ready\": context deadline exceeded"
Nov 04 01:32:17.520623 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.520590    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m" event={"ID":"9112be4e-695a-4087-a364-fd290ae66db3","Type":"ContainerDied","Data":"1b2831b534c48608357392707c3f159513d62db3019c9a603893f08e0d0290ef"}
Nov 04 01:32:17.520764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.520646    4434 scope.go:122] "RemoveContainer" containerID="39a8834311af0a39e30239583c521a7e6d0740ed13311f08313ea13ca072caeb"
Nov 04 01:32:17.520764 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.520609    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"
Nov 04 01:32:17.703637 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.703602    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"]
Nov 04 01:32:17.713933 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:17.713909    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-678aaf/simulator-w6-678aaf-kserve-7dfcc58ccb-w646m"]
Nov 04 01:32:19.473857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:19.473825    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9112be4e-695a-4087-a364-fd290ae66db3" path="/var/lib/kubelet/pods/9112be4e-695a-4087-a364-fd290ae66db3/volumes"
Nov 04 01:32:21.401930 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:21.401896    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:21.897349 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:32:21.897313    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:32:23.784610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:23.784531    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:23.997206 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:23.997173    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:24.238265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:24.238216    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:24.671788 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:24.671760    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:27.072267 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:27.072230    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:28.235682 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:28.235631    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:29.328134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:29.328084    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:29.540641 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:29.540604    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:29.681863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:29.681824    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"]
Nov 04 01:32:29.856425 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:29.856394    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:32.813951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:32.813918    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:32.997744 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:32.997716    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:33.575904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:33.575868    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:33.783423 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:33.783391    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:33.965060 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:33.965030    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:37.215881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:37.215843    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:37.418477 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:37.418441    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:37.831615 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:37.831578    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:39.594497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:39.594458    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:39.906172 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:39.906141    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:40.114318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:40.114288    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:40.311154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:40.311069    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:40.566583 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:40.566503    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:44.702077 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:44.702041    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main" containerID="cri-o://1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0" gracePeriod=45
Nov 04 01:32:44.982135 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:44.982103    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:32:45.104642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.104600    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm\") pod \"ea69d046-a993-4458-a52a-b30a2ff580d7\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") "
Nov 04 01:32:45.104842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.104794    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache\") pod \"ea69d046-a993-4458-a52a-b30a2ff580d7\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") "
Nov 04 01:32:45.104842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.104827    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir\") pod \"ea69d046-a993-4458-a52a-b30a2ff580d7\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") "
Nov 04 01:32:45.105000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.104905    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs\" (UniqueName: \"kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs\") pod \"ea69d046-a993-4458-a52a-b30a2ff580d7\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") "
Nov 04 01:32:45.105000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.104933    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home\") pod \"ea69d046-a993-4458-a52a-b30a2ff580d7\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") "
Nov 04 01:32:45.105000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.104962    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd\" (UniqueName: \"kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd\") pod \"ea69d046-a993-4458-a52a-b30a2ff580d7\" (UID: \"ea69d046-a993-4458-a52a-b30a2ff580d7\") "
Nov 04 01:32:45.105165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.105087    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache" pod "ea69d046-a993-4458-a52a-b30a2ff580d7" (UID: "ea69d046-a993-4458-a52a-b30a2ff580d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:45.105446 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.105426    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:45.106012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.105855    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home" pod "ea69d046-a993-4458-a52a-b30a2ff580d7" (UID: "ea69d046-a993-4458-a52a-b30a2ff580d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:45.106210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.106192    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir" pod "ea69d046-a993-4458-a52a-b30a2ff580d7" (UID: "ea69d046-a993-4458-a52a-b30a2ff580d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:45.107821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.107781    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs" pod "ea69d046-a993-4458-a52a-b30a2ff580d7" (UID: "ea69d046-a993-4458-a52a-b30a2ff580d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:45.107821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.107798    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm" pod "ea69d046-a993-4458-a52a-b30a2ff580d7" (UID: "ea69d046-a993-4458-a52a-b30a2ff580d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:32:45.111918 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.111898    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd" pod "ea69d046-a993-4458-a52a-b30a2ff580d7" (UID: "ea69d046-a993-4458-a52a-b30a2ff580d7"). InnerVolumeSpecName "kube-api-access-jkhtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:45.207125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.207041    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:45.207125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.207071    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea69d046-a993-4458-a52a-b30a2ff580d7-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:45.207125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.207079    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:45.207125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.207089    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkhtd\" (UniqueName: \"kubernetes.io/projected/ea69d046-a993-4458-a52a-b30a2ff580d7-kube-api-access-jkhtd\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:45.207125 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.207100    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea69d046-a993-4458-a52a-b30a2ff580d7-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:45.667949 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.667915    4434 generic.go:356] "Generic (PLEG): container finished" podID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerID="1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0" exitCode=0
Nov 04 01:32:45.668109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.667980    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"
Nov 04 01:32:45.668109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.668011    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" event={"ID":"ea69d046-a993-4458-a52a-b30a2ff580d7","Type":"ContainerDied","Data":"1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0"}
Nov 04 01:32:45.668109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.668056    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w" event={"ID":"ea69d046-a993-4458-a52a-b30a2ff580d7","Type":"ContainerDied","Data":"a3e5f3dad2649c5ff932b232a8f075e318622a807f229252976be35d22b97085"}
Nov 04 01:32:45.668109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.668072    4434 scope.go:122] "RemoveContainer" containerID="1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0"
Nov 04 01:32:45.676823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.676807    4434 scope.go:122] "RemoveContainer" containerID="1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0"
Nov 04 01:32:45.677082 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:32:45.677064    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0\": container with ID starting with 1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0 not found: ID does not exist" containerID="1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0"
Nov 04 01:32:45.677144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.677087    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0"} err="failed to get container status \"1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0\": rpc error: code = NotFound desc = could not find container \"1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0\": container with ID starting with 1c276bab580d1bb27e17e1b14cfb1b4707c57017c6b0a875b6d602e2f37815d0 not found: ID does not exist"
Nov 04 01:32:45.788478 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.788354    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:45.838013 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.837984    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"]
Nov 04 01:32:45.887479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:45.887448    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-e993d7/simulator-w0-e993d7-kserve-6b95cc586b-kzr7w"]
Nov 04 01:32:47.221861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:47.221823    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:47.479754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:47.476915    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" path="/var/lib/kubelet/pods/ea69d046-a993-4458-a52a-b30a2ff580d7/volumes"
Nov 04 01:32:47.507147 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:47.507119    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:47.693690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:47.692937    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:48.146815 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:48.146782    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:50.467118 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.467088    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"]
Nov 04 01:32:50.467525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.467343    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" podUID="e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" containerName="maas-api" containerID="cri-o://52421b6914aed01ce4e1e92c263a3ca456dcdc3fbd9e1ca7db69018ac8f78ed5" gracePeriod=30
Nov 04 01:32:50.635534 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.635499    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:50.730045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.729966    4434 generic.go:356] "Generic (PLEG): container finished" podID="e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" containerID="52421b6914aed01ce4e1e92c263a3ca456dcdc3fbd9e1ca7db69018ac8f78ed5" exitCode=0
Nov 04 01:32:50.730320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.730003    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" event={"ID":"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0","Type":"ContainerDied","Data":"52421b6914aed01ce4e1e92c263a3ca456dcdc3fbd9e1ca7db69018ac8f78ed5"}
Nov 04 01:32:50.757423 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.757401    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:32:50.880195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.880158    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz\" (UniqueName: \"kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz\") pod \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") "
Nov 04 01:32:50.880541 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.880520    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs\") pod \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") "
Nov 04 01:32:50.880835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.880817    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\") pod \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") "
Nov 04 01:32:50.880983 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.880968    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca\") pod \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\" (UID: \"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0\") "
Nov 04 01:32:50.882891 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.882822    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca" pod "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" (UID: "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:32:50.886594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.886570    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs" pod "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" (UID: "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:50.896015 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.895986    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls" pod "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" (UID: "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:32:50.898267 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.898239    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz" pod "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" (UID: "e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0"). InnerVolumeSpecName "kube-api-access-dqmrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:32:50.982619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.982534    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:50.982619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.982566    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqmrz\" (UniqueName: \"kubernetes.io/projected/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-kube-api-access-dqmrz\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:50.982619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.982580    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:50.982619 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:50.982589    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:32:51.737057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:51.737029    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"
Nov 04 01:32:51.737505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:51.737049    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8" event={"ID":"e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0","Type":"ContainerDied","Data":"1053c8d8a9c6c351300f7101e585d33a6b06a2c2d6b2d8a2b7556009c18fb994"}
Nov 04 01:32:51.737505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:51.737111    4434 scope.go:122] "RemoveContainer" containerID="52421b6914aed01ce4e1e92c263a3ca456dcdc3fbd9e1ca7db69018ac8f78ed5"
Nov 04 01:32:51.898628 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:32:51.898591    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:32:51.907822 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:51.907796    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"]
Nov 04 01:32:51.917642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:51.917613    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-678aaf-57f575ccf6-2q7p8"]
Nov 04 01:32:52.284939 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:52.284860    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:52.541625 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:52.541549    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:52.737284 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:52.737230    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:53.241406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:53.241375    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:53.473714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:53.473658    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" path="/var/lib/kubelet/pods/e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0/volumes"
Nov 04 01:32:57.913049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:57.913018    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:59.433248 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.433216    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:59.636804 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.636729    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:32:59.695342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.695268    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"]
Nov 04 01:32:59.695489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.695471    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy" containerID="cri-o://61e94d4eba1826b5fc0fc28846e2c7bcf947403880454c64050251612997076a" gracePeriod=30
Nov 04 01:32:59.718286 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.718254    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"]
Nov 04 01:32:59.718506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.718484    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy" containerID="cri-o://3586ce424de1be66ad9c142fe509c22d656f979b0e87d0d647bc3e0ade25a3cc" gracePeriod=30
Nov 04 01:32:59.737109 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.737081    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"]
Nov 04 01:32:59.737302 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.737273    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" containerID="cri-o://f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f" gracePeriod=30
Nov 04 01:32:59.827747 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:32:59.827651    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:00.488413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:00.488380    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:00.670397 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:00.670361    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:01.905334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:01.905299    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:02.109714 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:02.109657    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:03.234450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:03.234417    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:03.579170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:03.579089    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:03.757295 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:03.757257    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:04.193959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.193901    4434 patch_prober.go:29] interesting pod/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:33:04.194134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.193989    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:33:04.835375 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.835345    4434 generic.go:356] "Generic (PLEG): container finished" podID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerID="3586ce424de1be66ad9c142fe509c22d656f979b0e87d0d647bc3e0ade25a3cc" exitCode=0
Nov 04 01:33:04.835811 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.835414    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" event={"ID":"4f68d84c-81d2-451e-a0c5-bddcfcc625d5","Type":"ContainerDied","Data":"3586ce424de1be66ad9c142fe509c22d656f979b0e87d0d647bc3e0ade25a3cc"}
Nov 04 01:33:04.837029 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.837008    4434 generic.go:356] "Generic (PLEG): container finished" podID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerID="61e94d4eba1826b5fc0fc28846e2c7bcf947403880454c64050251612997076a" exitCode=0
Nov 04 01:33:04.837119 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.837070    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" event={"ID":"c4bffc01-0d12-438e-aaec-7873e04b837d","Type":"ContainerDied","Data":"61e94d4eba1826b5fc0fc28846e2c7bcf947403880454c64050251612997076a"}
Nov 04 01:33:04.990802 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:04.990635    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:33:05.034442 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.034414    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:33:05.052079 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.052056    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:33:05.069308 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069284    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.069455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069315    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.069455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069353    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.069455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069385    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.069455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069404    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.069455 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069448    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.069742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069485    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.069742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069521    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.069742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069549    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.069742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069589    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.069742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069641    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.069742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069716    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069751    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069763    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069887    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069890    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069943    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069966    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.069983    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070014    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.070064 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070029    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:05.070538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070076    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.070538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070132    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.070538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070182    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data\") pod \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\" (UID: \"4f68d84c-81d2-451e-a0c5-bddcfcc625d5\") "
Nov 04 01:33:05.070538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070220    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket\") pod \"c4bffc01-0d12-438e-aaec-7873e04b837d\" (UID: \"c4bffc01-0d12-438e-aaec-7873e04b837d\") "
Nov 04 01:33:05.070810 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070782    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.070876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070814    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.070876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070831    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.070876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070846    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4bffc01-0d12-438e-aaec-7873e04b837d-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.070876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070862    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.071079 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070876    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.071079 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.070890    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.071182 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.071137    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.072254 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072225    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.072368 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072263    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:05.072520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072449    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.072520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072476    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.072520 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072501    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.072735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072593    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:05.072814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072791    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:05.072886 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.072870    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:05.073052 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.073033    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq" pod "4f68d84c-81d2-451e-a0c5-bddcfcc625d5" (UID: "4f68d84c-81d2-451e-a0c5-bddcfcc625d5"). InnerVolumeSpecName "kube-api-access-w6rfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:05.073368 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.073344    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77" pod "c4bffc01-0d12-438e-aaec-7873e04b837d" (UID: "c4bffc01-0d12-438e-aaec-7873e04b837d"). InnerVolumeSpecName "kube-api-access-5gt77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:05.172133 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172100    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172150    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172200    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172277    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172373    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172392    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.172509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172418    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172448    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172500    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172546    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy\") pod \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\" (UID: \"9640fa9a-743b-4a7c-bc0b-88f2a1299c03\") "
Nov 04 01:33:05.172845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172652    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.172845 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172788    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.173011 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.172802    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:05.173111 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173092    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173116    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173134    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173148    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173168 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173164    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173178    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173192    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173206    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173221    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173235    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173248    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173262    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6rfq\" (UniqueName: \"kubernetes.io/projected/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-kube-api-access-w6rfq\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173275    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4bffc01-0d12-438e-aaec-7873e04b837d-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173289    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gt77\" (UniqueName: \"kubernetes.io/projected/c4bffc01-0d12-438e-aaec-7873e04b837d-kube-api-access-5gt77\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173303    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f68d84c-81d2-451e-a0c5-bddcfcc625d5-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.173898 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.173546    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.174878 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.174848    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:05.175069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.175050    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "kube-api-access-nlzrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:05.175154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.175127    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:05.175573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.175551    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token" pod "9640fa9a-743b-4a7c-bc0b-88f2a1299c03" (UID: "9640fa9a-743b-4a7c-bc0b-88f2a1299c03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:05.274733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.274643    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.274733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.274688    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.274733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.274699    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlzrg\" (UniqueName: \"kubernetes.io/projected/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-kube-api-access-nlzrg\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.274733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.274709    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.274733 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.274720    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9640fa9a-743b-4a7c-bc0b-88f2a1299c03-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:05.846595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.846564    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"
Nov 04 01:33:05.846595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.846578    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88" event={"ID":"4f68d84c-81d2-451e-a0c5-bddcfcc625d5","Type":"ContainerDied","Data":"63222f5acd8c94fe8b48b8cca6d78ee9a13179101d01baeeff2bcb311206aa15"}
Nov 04 01:33:05.847324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.846712    4434 scope.go:122] "RemoveContainer" containerID="3586ce424de1be66ad9c142fe509c22d656f979b0e87d0d647bc3e0ade25a3cc"
Nov 04 01:33:05.850246 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.850120    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb" event={"ID":"c4bffc01-0d12-438e-aaec-7873e04b837d","Type":"ContainerDied","Data":"6fee771903ed914b09b0c42ba05decb18b2d1af1a5d2756906342b322aa910f6"}
Nov 04 01:33:05.850246 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.850219    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"
Nov 04 01:33:05.859981 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.859951    4434 generic.go:356] "Generic (PLEG): container finished" podID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerID="f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f" exitCode=0
Nov 04 01:33:05.860115 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.860080    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" event={"ID":"9640fa9a-743b-4a7c-bc0b-88f2a1299c03","Type":"ContainerDied","Data":"f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f"}
Nov 04 01:33:05.860159 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.860133    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6" event={"ID":"9640fa9a-743b-4a7c-bc0b-88f2a1299c03","Type":"ContainerDied","Data":"2d5a18ddf4fad65272943d676d0cb16ad785c1eacc7a8070f4081af6d998abdd"}
Nov 04 01:33:05.860223 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.860209    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"
Nov 04 01:33:05.872693 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.872674    4434 scope.go:122] "RemoveContainer" containerID="61e94d4eba1826b5fc0fc28846e2c7bcf947403880454c64050251612997076a"
Nov 04 01:33:05.881989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.881962    4434 scope.go:122] "RemoveContainer" containerID="f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f"
Nov 04 01:33:05.890958 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.890937    4434 scope.go:122] "RemoveContainer" containerID="f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f"
Nov 04 01:33:05.891270 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:05.891242    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f\": container with ID starting with f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f not found: ID does not exist" containerID="f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f"
Nov 04 01:33:05.891345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.891274    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f"} err="failed to get container status \"f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f\": rpc error: code = NotFound desc = could not find container \"f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f\": container with ID starting with f830dff3134a7cbbde57e5296b0c864c4e46cbbedd88371773e74a0936f3dc8f not found: ID does not exist"
Nov 04 01:33:05.948855 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:05.948767    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:06.024122 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.024041    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"]
Nov 04 01:33:06.026121 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.026097    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-s29bb"]
Nov 04 01:33:06.125807 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.125765    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:06.137167 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.137135    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"]
Nov 04 01:33:06.142610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.142587    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-fbpv6"]
Nov 04 01:33:06.270757 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.270724    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"]
Nov 04 01:33:06.275033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:06.274945    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-678aaf-openshift-default-5b9b5966c5-khj88"]
Nov 04 01:33:07.473683 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:07.473639    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" path="/var/lib/kubelet/pods/4f68d84c-81d2-451e-a0c5-bddcfcc625d5/volumes"
Nov 04 01:33:07.474333 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:07.474311    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" path="/var/lib/kubelet/pods/9640fa9a-743b-4a7c-bc0b-88f2a1299c03/volumes"
Nov 04 01:33:07.474805 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:07.474786    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" path="/var/lib/kubelet/pods/c4bffc01-0d12-438e-aaec-7873e04b837d/volumes"
Nov 04 01:33:07.766849 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:07.766764    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:08.266057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:08.266006    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:08.757392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:08.757358    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:13.585644 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:13.585607    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:13.929123 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:13.929084    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:14.172676 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:14.172641    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:16.038194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:16.038157    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:16.287676 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:16.287637    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:18.232965 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.232921    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"]
Nov 04 01:33:18.233405 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.233102    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" podUID="7b514c36-7b50-44cd-bc14-e1d19296829b" containerName="maas-api" containerID="cri-o://a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97" gracePeriod=30
Nov 04 01:33:18.395631 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.395599    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:18.500867 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.500768    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:33:18.623471 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.623434    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca\") pod \"7b514c36-7b50-44cd-bc14-e1d19296829b\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") "
Nov 04 01:33:18.623691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.623569    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\") pod \"7b514c36-7b50-44cd-bc14-e1d19296829b\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") "
Nov 04 01:33:18.623691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.623604    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q\" (UniqueName: \"kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q\") pod \"7b514c36-7b50-44cd-bc14-e1d19296829b\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") "
Nov 04 01:33:18.623691 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.623628    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs\") pod \"7b514c36-7b50-44cd-bc14-e1d19296829b\" (UID: \"7b514c36-7b50-44cd-bc14-e1d19296829b\") "
Nov 04 01:33:18.625695 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.625068    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca" pod "7b514c36-7b50-44cd-bc14-e1d19296829b" (UID: "7b514c36-7b50-44cd-bc14-e1d19296829b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:18.656781 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.644853    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q" pod "7b514c36-7b50-44cd-bc14-e1d19296829b" (UID: "7b514c36-7b50-44cd-bc14-e1d19296829b"). InnerVolumeSpecName "kube-api-access-29v6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:18.656781 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.644845    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls" pod "7b514c36-7b50-44cd-bc14-e1d19296829b" (UID: "7b514c36-7b50-44cd-bc14-e1d19296829b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:18.656781 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.645511    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs" pod "7b514c36-7b50-44cd-bc14-e1d19296829b" (UID: "7b514c36-7b50-44cd-bc14-e1d19296829b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:18.725390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.725357    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b514c36-7b50-44cd-bc14-e1d19296829b-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.725390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.725388    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.725588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.725397    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29v6q\" (UniqueName: \"kubernetes.io/projected/7b514c36-7b50-44cd-bc14-e1d19296829b-kube-api-access-29v6q\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.725588 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.725407    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b514c36-7b50-44cd-bc14-e1d19296829b-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:18.929816 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.929784    4434 generic.go:356] "Generic (PLEG): container finished" podID="7b514c36-7b50-44cd-bc14-e1d19296829b" containerID="a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97" exitCode=0
Nov 04 01:33:18.929997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.929828    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"
Nov 04 01:33:18.929997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.929868    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" event={"ID":"7b514c36-7b50-44cd-bc14-e1d19296829b","Type":"ContainerDied","Data":"a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97"}
Nov 04 01:33:18.929997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.929927    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7" event={"ID":"7b514c36-7b50-44cd-bc14-e1d19296829b","Type":"ContainerDied","Data":"741169c52ea75a40085c150c7788f86706fc3634612ef02c6e137733179ec0de"}
Nov 04 01:33:18.929997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.929949    4434 scope.go:122] "RemoveContainer" containerID="a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97"
Nov 04 01:33:18.940407 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.940392    4434 scope.go:122] "RemoveContainer" containerID="a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97"
Nov 04 01:33:18.940674 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:18.940641    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97\": container with ID starting with a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97 not found: ID does not exist" containerID="a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97"
Nov 04 01:33:18.940721 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:18.940694    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97"} err="failed to get container status \"a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97\": rpc error: code = NotFound desc = could not find container \"a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97\": container with ID starting with a9c309c8a7b06a33ca123f21fb0fa5003d954d2766d9c52386bb24b6674eeb97 not found: ID does not exist"
Nov 04 01:33:19.101182 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:19.101150    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"]
Nov 04 01:33:19.106634 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:19.106608    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-e993d7-767cb57dfd-vqgz7"]
Nov 04 01:33:19.473617 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:19.473584    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b514c36-7b50-44cd-bc14-e1d19296829b" path="/var/lib/kubelet/pods/7b514c36-7b50-44cd-bc14-e1d19296829b/volumes"
Nov 04 01:33:19.785033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:19.784952    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:20.024799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:20.024763    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:20.357007 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:20.356971    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:21.027416 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:21.027386    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:21.356197 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:21.356107    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:21.669801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:21.669770    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:21.874535 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:21.874505    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:21.897990 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:21.897953    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:33:22.459132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:22.459097    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:25.000160 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:25.000129    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"]
Nov 04 01:33:25.000626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:25.000337    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" containerID="cri-o://d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9" gracePeriod=30
Nov 04 01:33:25.050774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:25.050738    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"]
Nov 04 01:33:25.050964 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:25.050942    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" containerID="cri-o://d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d" gracePeriod=30
Nov 04 01:33:26.712293 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:26.712261    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:33:26.712690 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:26.712326    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:33:27.006971 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.006893    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:27.554707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.554645    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:33:27.554882 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.554731    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:33:27.593058 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.593022    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:27.797427 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.797391    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:27.899475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.899437    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"]
Nov 04 01:33:27.899718 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:27.899641    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy" containerID="cri-o://f04995bf6b006b37860a1eb0329640f6cd5434a857e69a0f9f2ee0346394f428" gracePeriod=30
Nov 04 01:33:28.046077 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:28.046035    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:29.560023 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:29.559986    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:29.765205 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:29.765170    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:30.106126 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.106088    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:30.307351 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.307323    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:33:30.336122 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.336098    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:33:30.458558 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458521    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.458774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458585    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.458774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458624    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.458774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458658    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.458960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458812    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.458960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458914    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458971    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.458989    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.459073 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459010    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459089    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:30.459235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459095    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459160    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.459392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459314    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459181    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:30.459392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459364    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.459392 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459250    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459397    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459408    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459434    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459488    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459518    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459551    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket\") pod \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\" (UID: \"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd\") "
Nov 04 01:33:30.459594 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459557    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459614    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data\") pod \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\" (UID: \"9024f494-9faf-4e2b-9fe1-eb33b94ca551\") "
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.459797    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460163    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460185    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460202    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460217    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460231    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460233    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460246    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460261    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.460955 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460444    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.460955 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.460459    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.461503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.461418    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.461739 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.461690    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "kube-api-access-8862h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:30.462268 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.462244    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:30.462461 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.462421    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:30.462573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.462485    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "kube-api-access-vsv8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:30.462573 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.462501    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token" pod "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" (UID: "5c8ed7bb-946b-496f-ae42-a632f7b7d8cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:30.463143 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.463124    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:30.463318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.463298    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo" pod "9024f494-9faf-4e2b-9fe1-eb33b94ca551" (UID: "9024f494-9faf-4e2b-9fe1-eb33b94ca551"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:30.511018 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.510985    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:30.561318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561243    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561273    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561288    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561300    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561315    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9024f494-9faf-4e2b-9fe1-eb33b94ca551-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561328    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561340    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561355    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561368    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8862h\" (UniqueName: \"kubernetes.io/projected/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-kube-api-access-8862h\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561382    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsv8m\" (UniqueName: \"kubernetes.io/projected/9024f494-9faf-4e2b-9fe1-eb33b94ca551-kube-api-access-vsv8m\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:30.561877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:30.561397    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:31.029196 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.029163    4434 generic.go:356] "Generic (PLEG): container finished" podID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerID="d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d" exitCode=0
Nov 04 01:33:31.029406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.029212    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"
Nov 04 01:33:31.029406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.029248    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" event={"ID":"9024f494-9faf-4e2b-9fe1-eb33b94ca551","Type":"ContainerDied","Data":"d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d"}
Nov 04 01:33:31.029406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.029308    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892" event={"ID":"9024f494-9faf-4e2b-9fe1-eb33b94ca551","Type":"ContainerDied","Data":"b63596965fb415b591fcda8ffe8163ca9ea11e1bf37d7bba9c12ad4bf4625ea4"}
Nov 04 01:33:31.029406 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.029335    4434 scope.go:122] "RemoveContainer" containerID="d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d"
Nov 04 01:33:31.030721 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.030697    4434 generic.go:356] "Generic (PLEG): container finished" podID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerID="d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9" exitCode=0
Nov 04 01:33:31.030834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.030727    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" event={"ID":"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd","Type":"ContainerDied","Data":"d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9"}
Nov 04 01:33:31.030834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.030769    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"
Nov 04 01:33:31.030834 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.030790    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs" event={"ID":"5c8ed7bb-946b-496f-ae42-a632f7b7d8cd","Type":"ContainerDied","Data":"d51deb91a0b7ccc238fc8eb63463ad070e42bb8c5544411bc8c96d131754108b"}
Nov 04 01:33:31.039973 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.039952    4434 scope.go:122] "RemoveContainer" containerID="d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d"
Nov 04 01:33:31.040255 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:31.040229    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d\": container with ID starting with d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d not found: ID does not exist" containerID="d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d"
Nov 04 01:33:31.040354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.040267    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d"} err="failed to get container status \"d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d\": rpc error: code = NotFound desc = could not find container \"d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d\": container with ID starting with d686067c18cb8dd75978dcc2da09e1bdca15236d116d94fc6bee69c6e991192d not found: ID does not exist"
Nov 04 01:33:31.040354 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.040290    4434 scope.go:122] "RemoveContainer" containerID="d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9"
Nov 04 01:33:31.049852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.049832    4434 scope.go:122] "RemoveContainer" containerID="d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9"
Nov 04 01:33:31.050112 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:31.050094    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9\": container with ID starting with d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9 not found: ID does not exist" containerID="d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9"
Nov 04 01:33:31.050181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.050119    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9"} err="failed to get container status \"d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9\": rpc error: code = NotFound desc = could not find container \"d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9\": container with ID starting with d9baf66148bc727320f5a67512b44a854e477a60f90527a8606e185806086bf9 not found: ID does not exist"
Nov 04 01:33:31.198193 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.198162    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"]
Nov 04 01:33:31.202218 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.202195    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-s2892"]
Nov 04 01:33:31.312421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.312344    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"]
Nov 04 01:33:31.314111 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.314085    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-6bwfs"]
Nov 04 01:33:31.474673 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.474635    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" path="/var/lib/kubelet/pods/5c8ed7bb-946b-496f-ae42-a632f7b7d8cd/volumes"
Nov 04 01:33:31.475116 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:31.475102    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" path="/var/lib/kubelet/pods/9024f494-9faf-4e2b-9fe1-eb33b94ca551/volumes"
Nov 04 01:33:32.637335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:32.637286    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:33.045187 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.045155    4434 generic.go:356] "Generic (PLEG): container finished" podID="c479376a-8b56-48e9-8104-46518e09c0a5" containerID="f04995bf6b006b37860a1eb0329640f6cd5434a857e69a0f9f2ee0346394f428" exitCode=0
Nov 04 01:33:33.045325 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.045226    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" event={"ID":"c479376a-8b56-48e9-8104-46518e09c0a5","Type":"ContainerDied","Data":"f04995bf6b006b37860a1eb0329640f6cd5434a857e69a0f9f2ee0346394f428"}
Nov 04 01:33:33.171228 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.171206    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:33:33.187379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187325    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187360    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187383    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187399    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187425    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187472    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187509    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187573    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187650    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs\") pod \"c479376a-8b56-48e9-8104-46518e09c0a5\" (UID: \"c479376a-8b56-48e9-8104-46518e09c0a5\") "
Nov 04 01:33:33.187873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187742    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:33.187873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187750    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:33.188099 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187892    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:33.188099 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.187969    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:33.188211 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.188176    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:33.188271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.188253    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.188336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.188277    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.188336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.188293    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c479376a-8b56-48e9-8104-46518e09c0a5-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.188336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.188306    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.189587 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.189560    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:33.189709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.189583    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:33:33.189709 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.189647    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "kube-api-access-vgqnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:33.189792 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.189716    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token" pod "c479376a-8b56-48e9-8104-46518e09c0a5" (UID: "c479376a-8b56-48e9-8104-46518e09c0a5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:33.288894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.288863    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgqnx\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-kube-api-access-vgqnx\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.288894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.288889    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c479376a-8b56-48e9-8104-46518e09c0a5-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.289100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.288903    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.289100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.288916    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c479376a-8b56-48e9-8104-46518e09c0a5-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.289100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.288924    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c479376a-8b56-48e9-8104-46518e09c0a5-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:33.951869 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:33.951834    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:34.051927 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:34.051898    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"
Nov 04 01:33:34.052102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:34.051901    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp" event={"ID":"c479376a-8b56-48e9-8104-46518e09c0a5","Type":"ContainerDied","Data":"ae644bae3d34e17cb279b7820f4aeb8f6b9d9cb9b35cfa681b9517f2d3f5636d"}
Nov 04 01:33:34.052102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:34.052054    4434 scope.go:122] "RemoveContainer" containerID="f04995bf6b006b37860a1eb0329640f6cd5434a857e69a0f9f2ee0346394f428"
Nov 04 01:33:34.138061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:34.138030    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:34.269075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:34.268995    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"]
Nov 04 01:33:34.319633 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:34.319599    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e993d7-openshift-default-b7f7fcb7f-xhlgp"]
Nov 04 01:33:35.458439 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:35.458398    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"]
Nov 04 01:33:35.476039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:35.476005    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" path="/var/lib/kubelet/pods/c479376a-8b56-48e9-8104-46518e09c0a5/volumes"
Nov 04 01:33:35.624615 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:35.624571    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:37.128609 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:37.128573    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:38.864038 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:38.864011    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:39.167798 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:39.167767    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:39.371599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:39.371567    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:39.859995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:39.859955    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:42.442275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:42.442238    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:42.656234 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:42.656202    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:44.575922 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:44.575890    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:46.040894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:46.040857    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:46.213968 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:46.213931    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:46.401533 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:46.401502    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:49.214012 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.213974    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"]
Nov 04 01:33:49.214578 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214557    4434 cpu_manager.go:414] "containerMap: removing container" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214579    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81eda2bb-32a2-48dd-b3d0-9194f3170c1d" containerName="cleanup"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214609    4434 cpu_manager.go:414] "containerMap: removing container" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214615    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214622    4434 cpu_manager.go:414] "containerMap: removing container" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214627    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214641    4434 cpu_manager.go:414] "containerMap: removing container" podUID="7b514c36-7b50-44cd-bc14-e1d19296829b" containerName="maas-api"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214645    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b514c36-7b50-44cd-bc14-e1d19296829b" containerName="maas-api"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214653    4434 cpu_manager.go:414] "containerMap: removing container" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214657    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy"
Nov 04 01:33:49.214679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214679    4434 cpu_manager.go:414] "containerMap: removing container" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214687    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214702    4434 cpu_manager.go:414] "containerMap: removing container" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214709    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214726    4434 cpu_manager.go:414] "containerMap: removing container" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214732    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214744    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214756    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e10fad34-9218-48f1-863f-05b54184b28f" containerName="cleanup"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214776    4434 cpu_manager.go:414] "containerMap: removing container" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214783    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214797    4434 cpu_manager.go:414] "containerMap: removing container" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214804    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214819    4434 cpu_manager.go:414] "containerMap: removing container" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214825    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214842    4434 cpu_manager.go:414] "containerMap: removing container" podUID="e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" containerName="maas-api"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214848    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" containerName="maas-api"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214857    4434 cpu_manager.go:414] "containerMap: removing container" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214863    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214976    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="ea69d046-a993-4458-a52a-b30a2ff580d7" containerName="main"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214990    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="036e577b-422c-4d64-aad7-8d1a68f36417" containerName="cleanup"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.214999    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f68d84c-81d2-451e-a0c5-bddcfcc625d5" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215009    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1e50c1c-8e1c-495c-8373-4ea55cfcc0c0" containerName="maas-api"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215018    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4bffc01-0d12-438e-aaec-7873e04b837d" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215029    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="9112be4e-695a-4087-a364-fd290ae66db3" containerName="main"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215038    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="9024f494-9faf-4e2b-9fe1-eb33b94ca551" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215050    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b514c36-7b50-44cd-bc14-e1d19296829b" containerName="maas-api"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215057    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="c479376a-8b56-48e9-8104-46518e09c0a5" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215065    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c8ed7bb-946b-496f-ae42-a632f7b7d8cd" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215073    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="9640fa9a-743b-4a7c-bc0b-88f2a1299c03" containerName="istio-proxy"
Nov 04 01:33:49.215202 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.215083    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="814094e6-b400-482c-b461-3957063e72e3" containerName="main"
Nov 04 01:33:49.218364 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.218342    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.275924 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.275891    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"]
Nov 04 01:33:49.332607 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.332555    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:33:49.333214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.332775    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qb9zm\""
Nov 04 01:33:49.361072 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.361035    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvbcv\" (UniqueName: \"kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.361230 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.361115    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.361282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.361235    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.462582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.462541    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvbcv\" (UniqueName: \"kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.462814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.462637    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.462814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.462734    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.463281 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.463253    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.465015 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.464970    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.520624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.520588    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvbcv\" (UniqueName: \"kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv\") pod \"maas-api-revoke-sub-5453776f-mtv2s\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.531261 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.531237    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:49.912640 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:49.912618    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"]
Nov 04 01:33:49.914980 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:33:49.914950    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8640bb9f_b073_4ecc_b1c3_b76176f700b3.slice/crio-710bb10dd4986efc18fc167b105a7baebbf2216ce58c07e23d20f61c79b32934 WatchSource:0}: Error finding container 710bb10dd4986efc18fc167b105a7baebbf2216ce58c07e23d20f61c79b32934: Status 404 returned error can't find the container with id 710bb10dd4986efc18fc167b105a7baebbf2216ce58c07e23d20f61c79b32934
Nov 04 01:33:50.135420 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.135378    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s" event={"ID":"8640bb9f-b073-4ecc-b1c3-b76176f700b3","Type":"ContainerStarted","Data":"710bb10dd4986efc18fc167b105a7baebbf2216ce58c07e23d20f61c79b32934"}
Nov 04 01:33:50.272914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.272842    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:50.480926 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.480817    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" containerID="cri-o://8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416" gracePeriod=45
Nov 04 01:33:50.745507 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.745475    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:33:50.882106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882074    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home\") pod \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") "
Nov 04 01:33:50.882294 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882128    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache\") pod \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") "
Nov 04 01:33:50.882294 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882163    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir\") pod \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") "
Nov 04 01:33:50.882294 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882205    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs\" (UniqueName: \"kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs\") pod \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") "
Nov 04 01:33:50.882294 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882260    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm\") pod \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") "
Nov 04 01:33:50.882521 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882415    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw\" (UniqueName: \"kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw\") pod \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\" (UID: \"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea\") "
Nov 04 01:33:50.882769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882723    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache" pod "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" (UID: "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:50.882936 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.882915    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir" pod "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" (UID: "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:50.883075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.883051    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home" pod "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" (UID: "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:50.885156 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.885127    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs" pod "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" (UID: "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:50.885345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.885324    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw" pod "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" (UID: "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea"). InnerVolumeSpecName "kube-api-access-kzwsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:50.885345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.885339    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm" pod "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" (UID: "abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:50.984341 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.984303    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzwsw\" (UniqueName: \"kubernetes.io/projected/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-kube-api-access-kzwsw\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.984341 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.984335    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.984341 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.984348    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.984595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.984362    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.984595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.984375    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:50.984595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:50.984386    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:51.153291 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.153254    4434 generic.go:356] "Generic (PLEG): container finished" podID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerID="8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416" exitCode=0
Nov 04 01:33:51.153504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.153334    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"
Nov 04 01:33:51.153504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.153357    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" event={"ID":"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea","Type":"ContainerDied","Data":"8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416"}
Nov 04 01:33:51.153504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.153429    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" event={"ID":"abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea","Type":"ContainerDied","Data":"926d472b90c6a1e6a53fb97b97444e36aa1551368fce0ce1f8b51bdbc5b1d393"}
Nov 04 01:33:51.153504 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.153457    4434 scope.go:122] "RemoveContainer" containerID="8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416"
Nov 04 01:33:51.155176 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.155150    4434 generic.go:356] "Generic (PLEG): container finished" podID="8640bb9f-b073-4ecc-b1c3-b76176f700b3" containerID="4f90ba2f15f6f0fe16da8f29536bff690a060174f07b31c95b30aaaa96a8e118" exitCode=0
Nov 04 01:33:51.155283 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.155244    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s" event={"ID":"8640bb9f-b073-4ecc-b1c3-b76176f700b3","Type":"ContainerDied","Data":"4f90ba2f15f6f0fe16da8f29536bff690a060174f07b31c95b30aaaa96a8e118"}
Nov 04 01:33:51.163707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.163681    4434 scope.go:122] "RemoveContainer" containerID="8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416"
Nov 04 01:33:51.164088 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:51.164060    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416\": container with ID starting with 8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416 not found: ID does not exist" containerID="8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416"
Nov 04 01:33:51.164170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.164108    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416"} err="failed to get container status \"8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416\": rpc error: code = NotFound desc = could not find container \"8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416\": container with ID starting with 8a0167e7b4d9a4a277e8a0b16e01251b827f73fd91ed38f238d6df1322c32416 not found: ID does not exist"
Nov 04 01:33:51.435649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.435572    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"]
Nov 04 01:33:51.442207 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.442181    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4"]
Nov 04 01:33:51.474676 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.474632    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" path="/var/lib/kubelet/pods/abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea/volumes"
Nov 04 01:33:51.740726 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:51.740632    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e210ef/simulator-w5-e210ef-kserve-844bbc78c9-ztjv4" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main" probeResult="failure" output="Get \"https://10.134.0.42:8000/ready\": context deadline exceeded"
Nov 04 01:33:51.899090 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:33:51.899049    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:33:52.313227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.313203    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:52.399399 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.399368    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca\") pod \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") "
Nov 04 01:33:52.399568 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.399442    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca\") pod \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") "
Nov 04 01:33:52.399568 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.399500    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv\" (UniqueName: \"kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv\") pod \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\" (UID: \"8640bb9f-b073-4ecc-b1c3-b76176f700b3\") "
Nov 04 01:33:52.399795 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.399769    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca" pod "8640bb9f-b073-4ecc-b1c3-b76176f700b3" (UID: "8640bb9f-b073-4ecc-b1c3-b76176f700b3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:52.401422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.401400    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca" pod "8640bb9f-b073-4ecc-b1c3-b76176f700b3" (UID: "8640bb9f-b073-4ecc-b1c3-b76176f700b3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:52.401511 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.401481    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv" pod "8640bb9f-b073-4ecc-b1c3-b76176f700b3" (UID: "8640bb9f-b073-4ecc-b1c3-b76176f700b3"). InnerVolumeSpecName "kube-api-access-wvbcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:52.500951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.500880    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvbcv\" (UniqueName: \"kubernetes.io/projected/8640bb9f-b073-4ecc-b1c3-b76176f700b3-kube-api-access-wvbcv\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:52.500951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.500907    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-xks-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:52.500951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:52.500917    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8640bb9f-b073-4ecc-b1c3-b76176f700b3-maas-api-openshift-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:53.167905 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:53.167875    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"
Nov 04 01:33:53.167905 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:53.167894    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s" event={"ID":"8640bb9f-b073-4ecc-b1c3-b76176f700b3","Type":"ContainerDied","Data":"710bb10dd4986efc18fc167b105a7baebbf2216ce58c07e23d20f61c79b32934"}
Nov 04 01:33:53.168122 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:53.167948    4434 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="710bb10dd4986efc18fc167b105a7baebbf2216ce58c07e23d20f61c79b32934"
Nov 04 01:33:55.608075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:55.607990    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:58.058782 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.058750    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"]
Nov 04 01:33:58.059234 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.058932    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" podUID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" containerName="discovery" containerID="cri-o://13d7b8bc9a32ff7376c4cc52bce5681cf9f21e39acfe04a97fba0ca94f7fca59" gracePeriod=30
Nov 04 01:33:58.194420 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.194387    4434 generic.go:356] "Generic (PLEG): container finished" podID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" containerID="13d7b8bc9a32ff7376c4cc52bce5681cf9f21e39acfe04a97fba0ca94f7fca59" exitCode=0
Nov 04 01:33:58.194599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.194450    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" event={"ID":"981ba35c-3a27-4c04-bce1-490e5c6d1f0a","Type":"ContainerDied","Data":"13d7b8bc9a32ff7376c4cc52bce5681cf9f21e39acfe04a97fba0ca94f7fca59"}
Nov 04 01:33:58.317126 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.317064    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:33:58.461066 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461034    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461138    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461172    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461271 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461254    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461276    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461311    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461450 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461330    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs\") pod \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\" (UID: \"981ba35c-3a27-4c04-bce1-490e5c6d1f0a\") "
Nov 04 01:33:58.461677 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.461613    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:33:58.463827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.463791    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:58.463959 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.463936    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:58.464030 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.463961    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:58.464237 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.464220    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:33:58.464968 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.464946    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "kube-api-access-hkx8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:33:58.465051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.464986    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs" pod "981ba35c-3a27-4c04-bce1-490e5c6d1f0a" (UID: "981ba35c-3a27-4c04-bce1-490e5c6d1f0a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:33:58.563119 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563078    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkx8f\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-kube-api-access-hkx8f\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:58.563119 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563116    4434 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-ca-configmap\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:58.563360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563131    4434 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-kubeconfig\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:58.563360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563144    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:58.563360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563156    4434 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-istio-csr-dns-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:58.563360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563164    4434 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-local-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:58.563360 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:58.563174    4434 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/981ba35c-3a27-4c04-bce1-490e5c6d1f0a-cacerts\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:33:59.201413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.201383    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"
Nov 04 01:33:59.201902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.201423    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz" event={"ID":"981ba35c-3a27-4c04-bce1-490e5c6d1f0a","Type":"ContainerDied","Data":"7c4d755bdf97e37ed3fb7f84dd2d12d80f7d338ffad37cc7271d4b3a700b4b60"}
Nov 04 01:33:59.201902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.201493    4434 scope.go:122] "RemoveContainer" containerID="13d7b8bc9a32ff7376c4cc52bce5681cf9f21e39acfe04a97fba0ca94f7fca59"
Nov 04 01:33:59.363864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.363813    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:33:59.380648 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.380618    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"]
Nov 04 01:33:59.430444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.430406    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p9rpz"]
Nov 04 01:33:59.478345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.478254    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" path="/var/lib/kubelet/pods/981ba35c-3a27-4c04-bce1-490e5c6d1f0a/volumes"
Nov 04 01:33:59.560640 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:33:59.560601    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:00.811249 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:00.811217    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:05.248077 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:05.248038    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:06.859458 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:06.859424    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:07.125549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:07.125509    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:07.335011 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:07.334976    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:07.944506 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:07.944466    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:08.142337 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:08.142306    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:08.453952 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:08.453919    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:11.648842 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.648808    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"]
Nov 04 01:34:11.649252 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.649028    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" podUID="a922c3b3-2d94-438d-b0a0-3b15c97d0d02" containerName="maas-api" containerID="cri-o://dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3" gracePeriod=30
Nov 04 01:34:11.787628 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.787593    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:11.930296 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.930269    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:34:11.996342 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.996309    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd\" (UniqueName: \"kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd\") pod \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") "
Nov 04 01:34:11.996518 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.996403    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca\") pod \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") "
Nov 04 01:34:11.996518 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.996445    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs\") pod \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") "
Nov 04 01:34:11.996518 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.996475    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\") pod \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\" (UID: \"a922c3b3-2d94-438d-b0a0-3b15c97d0d02\") "
Nov 04 01:34:11.996844 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.996807    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca" pod "a922c3b3-2d94-438d-b0a0-3b15c97d0d02" (UID: "a922c3b3-2d94-438d-b0a0-3b15c97d0d02"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:11.997001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.996977    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:11.998613 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.998588    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd" pod "a922c3b3-2d94-438d-b0a0-3b15c97d0d02" (UID: "a922c3b3-2d94-438d-b0a0-3b15c97d0d02"). InnerVolumeSpecName "kube-api-access-x96sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:11.998738 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.998700    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs" pod "a922c3b3-2d94-438d-b0a0-3b15c97d0d02" (UID: "a922c3b3-2d94-438d-b0a0-3b15c97d0d02"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:11.998894 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:11.998873    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls" pod "a922c3b3-2d94-438d-b0a0-3b15c97d0d02" (UID: "a922c3b3-2d94-438d-b0a0-3b15c97d0d02"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:12.098174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.098134    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x96sd\" (UniqueName: \"kubernetes.io/projected/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-kube-api-access-x96sd\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:12.098174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.098166    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:12.098174 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.098182    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a922c3b3-2d94-438d-b0a0-3b15c97d0d02-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:12.265348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.265312    4434 generic.go:356] "Generic (PLEG): container finished" podID="a922c3b3-2d94-438d-b0a0-3b15c97d0d02" containerID="dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3" exitCode=0
Nov 04 01:34:12.265505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.265366    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"
Nov 04 01:34:12.265505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.265369    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" event={"ID":"a922c3b3-2d94-438d-b0a0-3b15c97d0d02","Type":"ContainerDied","Data":"dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3"}
Nov 04 01:34:12.265505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.265418    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6" event={"ID":"a922c3b3-2d94-438d-b0a0-3b15c97d0d02","Type":"ContainerDied","Data":"41ca26e005a07aa20c5ea31d216202da7df89059801e9e3f579840008b2b25c9"}
Nov 04 01:34:12.265505 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.265450    4434 scope.go:122] "RemoveContainer" containerID="dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3"
Nov 04 01:34:12.274642 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.274620    4434 scope.go:122] "RemoveContainer" containerID="dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3"
Nov 04 01:34:12.274924 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:34:12.274903    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3\": container with ID starting with dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3 not found: ID does not exist" containerID="dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3"
Nov 04 01:34:12.274995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.274939    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3"} err="failed to get container status \"dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3\": rpc error: code = NotFound desc = could not find container \"dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3\": container with ID starting with dafcec0e424dc0878f027b66c6c615699391216307323d54e06f652a3aeb66f3 not found: ID does not exist"
Nov 04 01:34:12.431011 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.430932    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"]
Nov 04 01:34:12.481080 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:12.481055    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-843be0-7c4b57c76c-tcvg6"]
Nov 04 01:34:13.472889 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:13.472854    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a922c3b3-2d94-438d-b0a0-3b15c97d0d02" path="/var/lib/kubelet/pods/a922c3b3-2d94-438d-b0a0-3b15c97d0d02/volumes"
Nov 04 01:34:13.546289 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:13.546253    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:13.732571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:13.732448    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:14.059089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:14.059012    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:14.246419 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:14.246378    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:14.429046 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:14.429014    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:14.902048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:14.902018    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:15.656303 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:15.656266    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:15.868960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:15.868925    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:16.006285 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:16.006201    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"]
Nov 04 01:34:16.006748 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:16.006403    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy" containerID="cri-o://7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf" gracePeriod=30
Nov 04 01:34:16.077604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:16.077567    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:18.997918 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:18.997888    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:21.261871 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.261844    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:34:21.315468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.315378    4434 generic.go:356] "Generic (PLEG): container finished" podID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerID="7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf" exitCode=0
Nov 04 01:34:21.315468 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.315448    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"
Nov 04 01:34:21.318056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.315481    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" event={"ID":"55f8f6a1-4e0e-4dc5-8c39-5dd939897433","Type":"ContainerDied","Data":"7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf"}
Nov 04 01:34:21.318056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.315550    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt" event={"ID":"55f8f6a1-4e0e-4dc5-8c39-5dd939897433","Type":"ContainerDied","Data":"2d7020dd6ddb70cac9cfd4218724f9f877c1031b80c5e766d1d98e4da0511bd5"}
Nov 04 01:34:21.318056 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.315576    4434 scope.go:122] "RemoveContainer" containerID="7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf"
Nov 04 01:34:21.326734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.326702    4434 scope.go:122] "RemoveContainer" containerID="7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf"
Nov 04 01:34:21.332054 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:34:21.332016    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf\": container with ID starting with 7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf not found: ID does not exist" containerID="7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf"
Nov 04 01:34:21.332210 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.332066    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf"} err="failed to get container status \"7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf\": rpc error: code = NotFound desc = could not find container \"7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf\": container with ID starting with 7a2a2ec29c2119d4a9093762178673e85365cd67bd55517234d5e246438ffbcf not found: ID does not exist"
Nov 04 01:34:21.392075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392038    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392075 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392084    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392137    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392160    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392195    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392349 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392341    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392371    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392397    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:21.392564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392411    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392460    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:21.392564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392514    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket\") pod \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\" (UID: \"55f8f6a1-4e0e-4dc5-8c39-5dd939897433\") "
Nov 04 01:34:21.392831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392544    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:21.392831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392776    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:21.392831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392804    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:21.392975 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392940    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.392975 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392958    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.392975 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392972    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.393134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.392987    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.393134 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.393000    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.394491 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.394463    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "kube-api-access-grwr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:21.394677 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.394638    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:21.394870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.394842    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:21.394870 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.394851    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo" pod "55f8f6a1-4e0e-4dc5-8c39-5dd939897433" (UID: "55f8f6a1-4e0e-4dc5-8c39-5dd939897433"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:34:21.493902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.493871    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.493902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.493900    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.494074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.493911    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grwr5\" (UniqueName: \"kubernetes.io/projected/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-kube-api-access-grwr5\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.494074 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.493920    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55f8f6a1-4e0e-4dc5-8c39-5dd939897433-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:21.736638 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.736598    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"]
Nov 04 01:34:21.748384 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.748355    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e210ef-openshift-default-bc95699cc-lktbt"]
Nov 04 01:34:21.809111 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:21.809077    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:21.900876 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:34:21.900835    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:34:22.076231 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:22.076135    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:22.348015 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:22.347933    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:22.766051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:22.766014    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:23.002873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.002840    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:23.248915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.248883    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"]
Nov 04 01:34:23.249275 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249262    4434 cpu_manager.go:414] "containerMap: removing container" podUID="a922c3b3-2d94-438d-b0a0-3b15c97d0d02" containerName="maas-api"
Nov 04 01:34:23.249320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249276    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a922c3b3-2d94-438d-b0a0-3b15c97d0d02" containerName="maas-api"
Nov 04 01:34:23.249320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249288    4434 cpu_manager.go:414] "containerMap: removing container" podUID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" containerName="discovery"
Nov 04 01:34:23.249320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249293    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" containerName="discovery"
Nov 04 01:34:23.249320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249303    4434 cpu_manager.go:414] "containerMap: removing container" podUID="8640bb9f-b073-4ecc-b1c3-b76176f700b3" containerName="revoke-keys"
Nov 04 01:34:23.249320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249308    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8640bb9f-b073-4ecc-b1c3-b76176f700b3" containerName="revoke-keys"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249324    4434 cpu_manager.go:414] "containerMap: removing container" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249329    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249343    4434 cpu_manager.go:414] "containerMap: removing container" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249348    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249406    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="abfe33ca-9ba1-42a1-a18f-8ccdcea7c8ea" containerName="main"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249417    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="a922c3b3-2d94-438d-b0a0-3b15c97d0d02" containerName="maas-api"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249423    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="981ba35c-3a27-4c04-bce1-490e5c6d1f0a" containerName="discovery"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249430    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="8640bb9f-b073-4ecc-b1c3-b76176f700b3" containerName="revoke-keys"
Nov 04 01:34:23.249474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.249437    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" containerName="istio-proxy"
Nov 04 01:34:23.253652 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.253636    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.309421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.309393    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"]
Nov 04 01:34:23.310814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.310791    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.310920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.310872    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.310920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.310904    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkhrw\" (UniqueName: \"kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.357839 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.357809    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:34:23.358172 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.357863    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:34:23.358172 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.357875    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:34:23.359680 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.359649    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qb9zm\""
Nov 04 01:34:23.411831 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.411803    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.411953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.411847    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.411953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.411869    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkhrw\" (UniqueName: \"kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.412445 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.412426    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.414175 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.414154    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.471054 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.471034    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkhrw\" (UniqueName: \"kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw\") pod \"maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.473820 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.473799    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55f8f6a1-4e0e-4dc5-8c39-5dd939897433" path="/var/lib/kubelet/pods/55f8f6a1-4e0e-4dc5-8c39-5dd939897433/volumes"
Nov 04 01:34:23.566583 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.566505    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:23.944616 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:23.944573    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"]
Nov 04 01:34:23.947652 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:34:23.947626    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f62a77c_e83d_4338_8e07_e69960a8f42a.slice/crio-84530a9dedbd0388bf461b2dfdbbb045ee48790575edcc7f15edf6ed034f46b1 WatchSource:0}: Error finding container 84530a9dedbd0388bf461b2dfdbbb045ee48790575edcc7f15edf6ed034f46b1: Status 404 returned error can't find the container with id 84530a9dedbd0388bf461b2dfdbbb045ee48790575edcc7f15edf6ed034f46b1
Nov 04 01:34:24.331755 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:24.331654    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58" event={"ID":"3f62a77c-e83d-4338-8e07-e69960a8f42a","Type":"ContainerStarted","Data":"11ab5df6d4a7270237832dbcc27af179a92bf26817b7799802826f697be4f1ef"}
Nov 04 01:34:24.331755 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:24.331729    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58" event={"ID":"3f62a77c-e83d-4338-8e07-e69960a8f42a","Type":"ContainerStarted","Data":"84530a9dedbd0388bf461b2dfdbbb045ee48790575edcc7f15edf6ed034f46b1"}
Nov 04 01:34:24.455743 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:24.455683    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58" podStartSLOduration=1.455651209 podStartE2EDuration="1.455651209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:24.454732188 +0000 UTC m=+2233.116872250" watchObservedRunningTime="2026-11-04 01:34:24.455651209 +0000 UTC m=+2233.117791271"
Nov 04 01:34:24.852863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:24.852829    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:25.337771 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:25.337738    4434 generic.go:356] "Generic (PLEG): container finished" podID="3f62a77c-e83d-4338-8e07-e69960a8f42a" containerID="11ab5df6d4a7270237832dbcc27af179a92bf26817b7799802826f697be4f1ef" exitCode=0
Nov 04 01:34:25.337946 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:25.337787    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58" event={"ID":"3f62a77c-e83d-4338-8e07-e69960a8f42a","Type":"ContainerDied","Data":"11ab5df6d4a7270237832dbcc27af179a92bf26817b7799802826f697be4f1ef"}
Nov 04 01:34:26.473130 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.473104    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:26.545102 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.545071    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca\") pod \"3f62a77c-e83d-4338-8e07-e69960a8f42a\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") "
Nov 04 01:34:26.545238 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.545126    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw\" (UniqueName: \"kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw\") pod \"3f62a77c-e83d-4338-8e07-e69960a8f42a\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") "
Nov 04 01:34:26.545304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.545283    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca\") pod \"3f62a77c-e83d-4338-8e07-e69960a8f42a\" (UID: \"3f62a77c-e83d-4338-8e07-e69960a8f42a\") "
Nov 04 01:34:26.545413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.545390    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca" pod "3f62a77c-e83d-4338-8e07-e69960a8f42a" (UID: "3f62a77c-e83d-4338-8e07-e69960a8f42a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:26.545569 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.545554    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-openshift-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:26.547159 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.547128    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw" pod "3f62a77c-e83d-4338-8e07-e69960a8f42a" (UID: "3f62a77c-e83d-4338-8e07-e69960a8f42a"). InnerVolumeSpecName "kube-api-access-jkhrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:26.547260 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.547226    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca" pod "3f62a77c-e83d-4338-8e07-e69960a8f42a" (UID: "3f62a77c-e83d-4338-8e07-e69960a8f42a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:26.647209 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.647184    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3f62a77c-e83d-4338-8e07-e69960a8f42a-maas-api-xks-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:26.647209 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:26.647208    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkhrw\" (UniqueName: \"kubernetes.io/projected/3f62a77c-e83d-4338-8e07-e69960a8f42a-kube-api-access-jkhrw\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:27.349902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:27.349865    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58" event={"ID":"3f62a77c-e83d-4338-8e07-e69960a8f42a","Type":"ContainerDied","Data":"84530a9dedbd0388bf461b2dfdbbb045ee48790575edcc7f15edf6ed034f46b1"}
Nov 04 01:34:27.350062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:27.349940    4434 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84530a9dedbd0388bf461b2dfdbbb045ee48790575edcc7f15edf6ed034f46b1"
Nov 04 01:34:27.350062 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:27.349877    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"
Nov 04 01:34:27.615445 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:27.615369    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"]
Nov 04 01:34:27.640358 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:27.640330    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"]
Nov 04 01:34:27.812921 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:27.812892    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:29.777901 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:29.777864    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:34.727893 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:34.727851    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:35.254877 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:35.254847    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:35.510467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:35.510380    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:39.242751 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.242712    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"]
Nov 04 01:34:39.243300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.243274    4434 cpu_manager.go:414] "containerMap: removing container" podUID="3f62a77c-e83d-4338-8e07-e69960a8f42a" containerName="revoke-keys"
Nov 04 01:34:39.243300 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.243302    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f62a77c-e83d-4338-8e07-e69960a8f42a" containerName="revoke-keys"
Nov 04 01:34:39.243459 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.243396    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f62a77c-e83d-4338-8e07-e69960a8f42a" containerName="revoke-keys"
Nov 04 01:34:39.246707 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.246690    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.302659 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.302633    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:34:39.307194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.307174    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:34:39.353401 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.353369    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:34:39.355358 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.355335    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"]
Nov 04 01:34:39.357564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.357535    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qb9zm\""
Nov 04 01:34:39.365101 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.365077    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xdbj\" (UniqueName: \"kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.365195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.365114    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.365248 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.365232    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.466840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.466800    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xdbj\" (UniqueName: \"kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.466840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.466843    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.467108 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.466907    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.467697 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.467650    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.469479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.469453    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.524655 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.524579    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xdbj\" (UniqueName: \"kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj\") pod \"maas-api-revoke-sub-779e08cc-4phcc\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.559437 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.559403    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:39.942923 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:39.942885    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"]
Nov 04 01:34:39.946934 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:34:39.946906    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a38b18b_9ce2_4e89_a7bf_1ce111d75eba.slice/crio-c3eaab3fadf2bd6b9219d0ab2782ee97ebcc3d5748c0e2cd0c7b778028de45ea WatchSource:0}: Error finding container c3eaab3fadf2bd6b9219d0ab2782ee97ebcc3d5748c0e2cd0c7b778028de45ea: Status 404 returned error can't find the container with id c3eaab3fadf2bd6b9219d0ab2782ee97ebcc3d5748c0e2cd0c7b778028de45ea
Nov 04 01:34:40.420066 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:40.420024    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc" event={"ID":"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba","Type":"ContainerStarted","Data":"f69d400a448655fd56cff8a0b9672869a0a024e423d170411555bee5be3c8e0d"}
Nov 04 01:34:40.420547 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:40.420086    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc" event={"ID":"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba","Type":"ContainerStarted","Data":"c3eaab3fadf2bd6b9219d0ab2782ee97ebcc3d5748c0e2cd0c7b778028de45ea"}
Nov 04 01:34:40.547618 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:40.547522    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc" podStartSLOduration=1.5475095190000001 podStartE2EDuration="1.547509519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:34:40.546628096 +0000 UTC m=+2249.208768159" watchObservedRunningTime="2026-11-04 01:34:40.547509519 +0000 UTC m=+2249.209649580"
Nov 04 01:34:41.426636 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:41.426597    4434 generic.go:356] "Generic (PLEG): container finished" podID="7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" containerID="f69d400a448655fd56cff8a0b9672869a0a024e423d170411555bee5be3c8e0d" exitCode=0
Nov 04 01:34:41.427165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:41.426689    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc" event={"ID":"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba","Type":"ContainerDied","Data":"f69d400a448655fd56cff8a0b9672869a0a024e423d170411555bee5be3c8e0d"}
Nov 04 01:34:42.570179 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.570153    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:42.635144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.635095    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" containerID="cri-o://5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c" gracePeriod=45
Nov 04 01:34:42.639363 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.639333    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": EOF"
Nov 04 01:34:42.659388 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.659359    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" containerID="cri-o://d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3" gracePeriod=45
Nov 04 01:34:42.704208 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.704181    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj\" (UniqueName: \"kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj\") pod \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") "
Nov 04 01:34:42.704312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.704231    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca\") pod \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") "
Nov 04 01:34:42.704312 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.704271    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca\") pod \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\" (UID: \"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba\") "
Nov 04 01:34:42.704725 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.704697    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca" pod "7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" (UID: "7a38b18b-9ce2-4e89-a7bf-1ce111d75eba"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:34:42.706218 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.706194    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca" pod "7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" (UID: "7a38b18b-9ce2-4e89-a7bf-1ce111d75eba"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:42.706320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.706304    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj" pod "7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" (UID: "7a38b18b-9ce2-4e89-a7bf-1ce111d75eba"). InnerVolumeSpecName "kube-api-access-4xdbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:42.805460 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.805420    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xdbj\" (UniqueName: \"kubernetes.io/projected/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-kube-api-access-4xdbj\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:42.805460 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.805460    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-xks-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:42.805679 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.805475    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba-maas-api-openshift-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:42.903766 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.903742    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:34:42.913987 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:42.913968    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:34:43.006928 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.006848    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm\") pod \"353b1b1b-6900-4caf-88c4-932744856777\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") "
Nov 04 01:34:43.007087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.006936    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home\") pod \"353b1b1b-6900-4caf-88c4-932744856777\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") "
Nov 04 01:34:43.007087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.006964    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache\") pod \"353b1b1b-6900-4caf-88c4-932744856777\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") "
Nov 04 01:34:43.007087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007025    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir\") pod \"353b1b1b-6900-4caf-88c4-932744856777\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") "
Nov 04 01:34:43.007087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007069    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs\" (UniqueName: \"kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs\") pod \"353b1b1b-6900-4caf-88c4-932744856777\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") "
Nov 04 01:34:43.007285 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007103    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn\" (UniqueName: \"kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn\") pod \"353b1b1b-6900-4caf-88c4-932744856777\" (UID: \"353b1b1b-6900-4caf-88c4-932744856777\") "
Nov 04 01:34:43.007285 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007226    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home" pod "353b1b1b-6900-4caf-88c4-932744856777" (UID: "353b1b1b-6900-4caf-88c4-932744856777"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.007490 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007470    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.007621 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007571    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache" pod "353b1b1b-6900-4caf-88c4-932744856777" (UID: "353b1b1b-6900-4caf-88c4-932744856777"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.007621 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.007604    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir" pod "353b1b1b-6900-4caf-88c4-932744856777" (UID: "353b1b1b-6900-4caf-88c4-932744856777"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.009451 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.009426    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm" pod "353b1b1b-6900-4caf-88c4-932744856777" (UID: "353b1b1b-6900-4caf-88c4-932744856777"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.009548 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.009453    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn" pod "353b1b1b-6900-4caf-88c4-932744856777" (UID: "353b1b1b-6900-4caf-88c4-932744856777"). InnerVolumeSpecName "kube-api-access-ks2mn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:43.009620 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.009602    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs" pod "353b1b1b-6900-4caf-88c4-932744856777" (UID: "353b1b1b-6900-4caf-88c4-932744856777"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:43.108591 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108561    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home\") pod \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") "
Nov 04 01:34:43.108801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108627    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs\" (UniqueName: \"kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs\") pod \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") "
Nov 04 01:34:43.108801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108694    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp\" (UniqueName: \"kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp\") pod \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") "
Nov 04 01:34:43.108801 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108718    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm\") pod \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") "
Nov 04 01:34:43.109001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108816    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache\") pod \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") "
Nov 04 01:34:43.109001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108863    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir\") pod \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\" (UID: \"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8\") "
Nov 04 01:34:43.109001 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.108882    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home" pod "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" (UID: "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.109183 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109153    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache" pod "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" (UID: "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.109304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109241    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir" pod "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" (UID: "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.109304 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109288    4434 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-home\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.109421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109308    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/353b1b1b-6900-4caf-88c4-932744856777-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.109421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109326    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ks2mn\" (UniqueName: \"kubernetes.io/projected/353b1b1b-6900-4caf-88c4-932744856777-kube-api-access-ks2mn\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.109421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109341    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.109421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109354    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.109421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109368    4434 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-model-cache\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.109421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.109384    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/353b1b1b-6900-4caf-88c4-932744856777-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.110827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.110799    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs" pod "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" (UID: "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:34:43.110827 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.110808    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm" pod "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" (UID: "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:34:43.110960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.110841    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp" pod "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" (UID: "d96fe731-d4cd-4c29-86a4-1e6175b0b7a8"). InnerVolumeSpecName "kube-api-access-zbxlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:34:43.210734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.210705    4434 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tls-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.210734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.210730    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbxlp\" (UniqueName: \"kubernetes.io/projected/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-kube-api-access-zbxlp\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.210734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.210740    4434 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-dshm\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.210953 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.210748    4434 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8-tmp-dir\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:34:43.437497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.437465    4434 generic.go:356] "Generic (PLEG): container finished" podID="353b1b1b-6900-4caf-88c4-932744856777" containerID="5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c" exitCode=0
Nov 04 01:34:43.437716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.437509    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"
Nov 04 01:34:43.437716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.437555    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" event={"ID":"353b1b1b-6900-4caf-88c4-932744856777","Type":"ContainerDied","Data":"5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c"}
Nov 04 01:34:43.437716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.437597    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl" event={"ID":"353b1b1b-6900-4caf-88c4-932744856777","Type":"ContainerDied","Data":"54b93ad153e0d636f5a800b1c80f0b5e66970a4f1c53443f8822a85dc5b8e229"}
Nov 04 01:34:43.437716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.437612    4434 scope.go:122] "RemoveContainer" containerID="5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c"
Nov 04 01:34:43.439100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.439076    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"
Nov 04 01:34:43.439100 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.439094    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc" event={"ID":"7a38b18b-9ce2-4e89-a7bf-1ce111d75eba","Type":"ContainerDied","Data":"c3eaab3fadf2bd6b9219d0ab2782ee97ebcc3d5748c0e2cd0c7b778028de45ea"}
Nov 04 01:34:43.439298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.439131    4434 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3eaab3fadf2bd6b9219d0ab2782ee97ebcc3d5748c0e2cd0c7b778028de45ea"
Nov 04 01:34:43.440634 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.440612    4434 generic.go:356] "Generic (PLEG): container finished" podID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerID="d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3" exitCode=0
Nov 04 01:34:43.440762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.440649    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" event={"ID":"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8","Type":"ContainerDied","Data":"d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3"}
Nov 04 01:34:43.440762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.440685    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"
Nov 04 01:34:43.440762 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.440694    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" event={"ID":"d96fe731-d4cd-4c29-86a4-1e6175b0b7a8","Type":"ContainerDied","Data":"d8fcb001c57612ecd2fb3777c64ffbf0ba532323b19eb3b466217b6e471389d3"}
Nov 04 01:34:43.447390 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.447375    4434 scope.go:122] "RemoveContainer" containerID="5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c"
Nov 04 01:34:43.447769 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:34:43.447729    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c\": container with ID starting with 5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c not found: ID does not exist" containerID="5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c"
Nov 04 01:34:43.447864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.447783    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c"} err="failed to get container status \"5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c\": rpc error: code = NotFound desc = could not find container \"5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c\": container with ID starting with 5fd247be3a377827f48c9d0fb3accfd62186e62b8b2ef363d2c8f33a708b488c not found: ID does not exist"
Nov 04 01:34:43.447864 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.447806    4434 scope.go:122] "RemoveContainer" containerID="d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3"
Nov 04 01:34:43.455700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.455660    4434 scope.go:122] "RemoveContainer" containerID="d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3"
Nov 04 01:34:43.455938 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:34:43.455920    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3\": container with ID starting with d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3 not found: ID does not exist" containerID="d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3"
Nov 04 01:34:43.455992 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.455943    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3"} err="failed to get container status \"d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3\": rpc error: code = NotFound desc = could not find container \"d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3\": container with ID starting with d3eb075b5f9f70b2a2c5cdad6d86a853c977ef7c9eabe21d4737cb11e2b0ded3 not found: ID does not exist"
Nov 04 01:34:43.610158 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.610121    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"]
Nov 04 01:34:43.613065 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.613039    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq"]
Nov 04 01:34:43.776528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.776446    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"]
Nov 04 01:34:43.782008 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.781974    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-2fd27d/distinct-2-w2-2fd27d-kserve-fdf689585-jfbcl"]
Nov 04 01:34:43.890335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:43.890296    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-2fd27d/simulator-w2-2fd27d-kserve-84fb4f79b6-hgxvq" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 04 01:34:45.473545 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:45.473512    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="353b1b1b-6900-4caf-88c4-932744856777" path="/var/lib/kubelet/pods/353b1b1b-6900-4caf-88c4-932744856777/volumes"
Nov 04 01:34:45.473975 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:45.473955    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" path="/var/lib/kubelet/pods/d96fe731-d4cd-4c29-86a4-1e6175b0b7a8/volumes"
Nov 04 01:34:48.169114 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:48.169077    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:48.357411 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:48.357378    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:48.596298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:48.596224    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:51.897005 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:34:51.896961    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:34:54.339282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:54.339245    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:54.662151 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:54.662120    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:56.241026 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:56.240979    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:56.517462 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:56.517391    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:56.721419 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:56.721383    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"]
Nov 04 01:34:56.771375 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:56.771303    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w2-2fd27d-2rg58"]
Nov 04 01:34:57.165740 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:57.165714    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:34:57.472620 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:34:57.472548    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f62a77c-e83d-4338-8e07-e69960a8f42a" path="/var/lib/kubelet/pods/3f62a77c-e83d-4338-8e07-e69960a8f42a/volumes"
Nov 04 01:35:03.886194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:03.886150    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:04.125880 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:04.125834    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:04.216308 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:04.216220    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"]
Nov 04 01:35:04.216475 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:04.216433    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" containerID="cri-o://4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d" gracePeriod=30
Nov 04 01:35:04.365434 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:04.365397    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:06.345282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:06.345224    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:06.596547 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:06.596460    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:07.462089 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:07.462051    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:08.478525 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:08.478491    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:08.892071 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:08.892032    4434 patch_prober.go:29] interesting pod/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 04 01:35:08.892246 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:08.892113    4434 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 04 01:35:09.463110 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.463086    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:35:09.565946 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.565865    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.565946 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.565921    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.565948    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.565977    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566029    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566109    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566148    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566206    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566206    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:09.566467 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566285    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs\") pod \"f393871e-3490-4f66-86c1-dff15707249e\" (UID: \"f393871e-3490-4f66-86c1-dff15707249e\") "
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566493    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566608    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566642    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566817    4434 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-data\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566839    4434 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-credential-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566854    4434 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-socket\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.566914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.566821    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:09.568372 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.568347    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "kube-api-access-vxbsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:09.568476 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.568373    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:09.568895 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.568881    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 04 01:35:09.568948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.568919    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo" pod "f393871e-3490-4f66-86c1-dff15707249e" (UID: "f393871e-3490-4f66-86c1-dff15707249e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 04 01:35:09.585884 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.585856    4434 generic.go:356] "Generic (PLEG): container finished" podID="f393871e-3490-4f66-86c1-dff15707249e" containerID="4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d" exitCode=0
Nov 04 01:35:09.585990 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.585905    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"
Nov 04 01:35:09.585990 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.585934    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" event={"ID":"f393871e-3490-4f66-86c1-dff15707249e","Type":"ContainerDied","Data":"4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d"}
Nov 04 01:35:09.586067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.585991    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl" event={"ID":"f393871e-3490-4f66-86c1-dff15707249e","Type":"ContainerDied","Data":"5de45195575b29ddce93e33d1457770624eab428205fdb7873f7f8059e38abdf"}
Nov 04 01:35:09.586067 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.586013    4434 scope.go:122] "RemoveContainer" containerID="4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d"
Nov 04 01:35:09.594571 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.594553    4434 scope.go:122] "RemoveContainer" containerID="4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d"
Nov 04 01:35:09.594882 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:35:09.594865    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d\": container with ID starting with 4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d not found: ID does not exist" containerID="4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d"
Nov 04 01:35:09.594954 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.594894    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d"} err="failed to get container status \"4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d\": rpc error: code = NotFound desc = could not find container \"4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d\": container with ID starting with 4142ab5270caf063dde87b6f40a6bf9283ce75f36ba70130a8c2c46ff396cd4d not found: ID does not exist"
Nov 04 01:35:09.668145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.668111    4434 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-istio-envoy\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.668145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.668141    4434 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-istio-token\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.668145 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.668151    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxbsv\" (UniqueName: \"kubernetes.io/projected/f393871e-3490-4f66-86c1-dff15707249e-kube-api-access-vxbsv\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.668367 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.668161    4434 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f393871e-3490-4f66-86c1-dff15707249e-istiod-ca-cert\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.668367 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.668170    4434 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f393871e-3490-4f66-86c1-dff15707249e-istio-podinfo\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.668367 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.668179    4434 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f393871e-3490-4f66-86c1-dff15707249e-workload-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:09.806497 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.806460    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"]
Nov 04 01:35:09.814336 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:09.814283    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-2fd27d-openshift-default-77cb48fb59-zwqtl"]
Nov 04 01:35:11.474416 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:11.474385    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f393871e-3490-4f66-86c1-dff15707249e" path="/var/lib/kubelet/pods/f393871e-3490-4f66-86c1-dff15707249e/volumes"
Nov 04 01:35:13.028538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:13.028505    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:13.206682 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:13.206620    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:13.396552 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:13.396523    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:21.895150 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:35:21.895109    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:35:38.123626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.123539    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"]
Nov 04 01:35:38.124162 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124119    4434 cpu_manager.go:414] "containerMap: removing container" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy"
Nov 04 01:35:38.124162 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124138    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy"
Nov 04 01:35:38.124301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124167    4434 cpu_manager.go:414] "containerMap: removing container" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main"
Nov 04 01:35:38.124301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124175    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main"
Nov 04 01:35:38.124301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124186    4434 cpu_manager.go:414] "containerMap: removing container" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main"
Nov 04 01:35:38.124301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124194    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main"
Nov 04 01:35:38.124301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124222    4434 cpu_manager.go:414] "containerMap: removing container" podUID="7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" containerName="revoke-keys"
Nov 04 01:35:38.124301 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124230    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" containerName="revoke-keys"
Nov 04 01:35:38.124603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124313    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="353b1b1b-6900-4caf-88c4-932744856777" containerName="main"
Nov 04 01:35:38.124603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124334    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" containerName="revoke-keys"
Nov 04 01:35:38.124603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124349    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="f393871e-3490-4f66-86c1-dff15707249e" containerName="istio-proxy"
Nov 04 01:35:38.124603 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.124361    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="d96fe731-d4cd-4c29-86a4-1e6175b0b7a8" containerName="main"
Nov 04 01:35:38.127410 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.127390    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.182604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.182572    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"]
Nov 04 01:35:38.230849 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.230816    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:35:38.230849 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.230832    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 04 01:35:38.231057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.230855    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:35:38.231057 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.230924    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-qb9zm\""
Nov 04 01:35:38.237328 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.237299    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sgdg\" (UniqueName: \"kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.237447 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.237366    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.237496 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.237451    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.339051 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.339013    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sgdg\" (UniqueName: \"kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.339221 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.339067    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.339221 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.339112    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.339875 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.339855    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.341683 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.341625    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.402910 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.402832    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sgdg\" (UniqueName: \"kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg\") pod \"maas-api-revoke-sub-85237e45-jrf5q\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.440140 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.440111    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:38.814036 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:38.813953    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"]
Nov 04 01:35:38.817801 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:35:38.817773    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74148ccf_c040_4c5c_9617_85d34cd255b6.slice/crio-3b769d2fb98ed77c339cb525e7e020759ce9ef90174f649a8bc298f355cfa9ad WatchSource:0}: Error finding container 3b769d2fb98ed77c339cb525e7e020759ce9ef90174f649a8bc298f355cfa9ad: Status 404 returned error can't find the container with id 3b769d2fb98ed77c339cb525e7e020759ce9ef90174f649a8bc298f355cfa9ad
Nov 04 01:35:39.731242 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:39.731204    4434 generic.go:356] "Generic (PLEG): container finished" podID="74148ccf-c040-4c5c-9617-85d34cd255b6" containerID="76b213754c3f432eb01da3c3219e25eb7562f6904e7b0bea58f732f2374c3785" exitCode=0
Nov 04 01:35:39.731621 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:39.731287    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q" event={"ID":"74148ccf-c040-4c5c-9617-85d34cd255b6","Type":"ContainerDied","Data":"76b213754c3f432eb01da3c3219e25eb7562f6904e7b0bea58f732f2374c3785"}
Nov 04 01:35:39.731621 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:39.731339    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q" event={"ID":"74148ccf-c040-4c5c-9617-85d34cd255b6","Type":"ContainerStarted","Data":"3b769d2fb98ed77c339cb525e7e020759ce9ef90174f649a8bc298f355cfa9ad"}
Nov 04 01:35:40.864379 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.864352    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:40.965233 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.965194    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca\") pod \"74148ccf-c040-4c5c-9617-85d34cd255b6\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") "
Nov 04 01:35:40.965404 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.965291    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg\" (UniqueName: \"kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg\") pod \"74148ccf-c040-4c5c-9617-85d34cd255b6\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") "
Nov 04 01:35:40.965404 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.965351    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca\") pod \"74148ccf-c040-4c5c-9617-85d34cd255b6\" (UID: \"74148ccf-c040-4c5c-9617-85d34cd255b6\") "
Nov 04 01:35:40.965626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.965596    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca" pod "74148ccf-c040-4c5c-9617-85d34cd255b6" (UID: "74148ccf-c040-4c5c-9617-85d34cd255b6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:35:40.967343 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.967322    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca" pod "74148ccf-c040-4c5c-9617-85d34cd255b6" (UID: "74148ccf-c040-4c5c-9617-85d34cd255b6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:35:40.967421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:40.967345    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg" pod "74148ccf-c040-4c5c-9617-85d34cd255b6" (UID: "74148ccf-c040-4c5c-9617-85d34cd255b6"). InnerVolumeSpecName "kube-api-access-5sgdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:35:41.067039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:41.066956    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-xks-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:41.067039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:41.066988    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74148ccf-c040-4c5c-9617-85d34cd255b6-maas-api-openshift-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:41.067039 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:41.066999    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sgdg\" (UniqueName: \"kubernetes.io/projected/74148ccf-c040-4c5c-9617-85d34cd255b6-kube-api-access-5sgdg\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:35:41.741232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:41.741151    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q" event={"ID":"74148ccf-c040-4c5c-9617-85d34cd255b6","Type":"ContainerDied","Data":"3b769d2fb98ed77c339cb525e7e020759ce9ef90174f649a8bc298f355cfa9ad"}
Nov 04 01:35:41.741232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:41.741202    4434 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b769d2fb98ed77c339cb525e7e020759ce9ef90174f649a8bc298f355cfa9ad"
Nov 04 01:35:41.741232 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:41.741163    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"
Nov 04 01:35:47.980041 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:47.980004    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:48.151600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:48.151567    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:50.898621 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:50.898576    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:35:51.897276 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:35:51.897238    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:35:53.884629 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:35:53.884595    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:08.748735 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:36:08.748699    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:12.777010 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:36:12.776976    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:36:21.896931 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:36:21.896888    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:36:51.895228 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:36:51.895185    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:37:09.102287 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:37:09.102199    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:09.368194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:37:09.368105    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:18.226348 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:37:18.226317    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:18.392395 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:37:18.392357    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:21.896400 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:37:21.896358    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:37:39.861977 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:37:39.861946    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:40.055421 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:37:40.055385    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:37:51.896119 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:37:51.896075    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:38:09.512374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:38:09.512341    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:21.896416 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:38:21.896380    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:38:27.372424 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:38:27.372344    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:38:51.896591 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:38:51.896551    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:38:53.061088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:38:53.061050    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"]
Nov 04 01:38:53.111306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:38:53.111275    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5453776f-mtv2s"]
Nov 04 01:38:53.473308 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:38:53.473271    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8640bb9f-b073-4ecc-b1c3-b76176f700b3" path="/var/lib/kubelet/pods/8640bb9f-b073-4ecc-b1c3-b76176f700b3/volumes"
Nov 04 01:39:09.442794 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:39:09.442765    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:09.620750 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:39:09.620720    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:09.858370 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:39:09.858276    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:39:21.896540 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:39:21.896497    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:39:43.053198 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:39:43.053159    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"]
Nov 04 01:39:43.102998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:39:43.102967    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-779e08cc-4phcc"]
Nov 04 01:39:43.474144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:39:43.474106    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a38b18b-9ce2-4e89-a7bf-1ce111d75eba" path="/var/lib/kubelet/pods/7a38b18b-9ce2-4e89-a7bf-1ce111d75eba/volumes"
Nov 04 01:39:51.896762 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:39:51.896723    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:40:10.142821 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:40:10.142758    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:10.316310 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:40:10.316276    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:40:12.259200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:40:12.259163    4434 scope.go:122] "RemoveContainer" containerID="4f90ba2f15f6f0fe16da8f29536bff690a060174f07b31c95b30aaaa96a8e118"
Nov 04 01:40:21.896552 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:40:21.896504    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:40:41.046993 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:40:41.046945    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"]
Nov 04 01:40:41.097265 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:40:41.097224    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jrf5q"]
Nov 04 01:40:41.472780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:40:41.472740    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74148ccf-c040-4c5c-9617-85d34cd255b6" path="/var/lib/kubelet/pods/74148ccf-c040-4c5c-9617-85d34cd255b6/volumes"
Nov 04 01:40:51.897011 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:40:51.896968    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:41:10.498771 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:41:10.498728    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:41:12.283937 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:41:12.283903    4434 scope.go:122] "RemoveContainer" containerID="f69d400a448655fd56cff8a0b9672869a0a024e423d170411555bee5be3c8e0d"
Nov 04 01:41:12.292054 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:41:12.292022    4434 scope.go:122] "RemoveContainer" containerID="11ab5df6d4a7270237832dbcc27af179a92bf26817b7799802826f697be4f1ef"
Nov 04 01:41:21.895593 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:41:21.895549    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:41:28.173578 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:41:28.173498    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:41:51.896305 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:41:51.896256    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:42:10.837653 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:10.837606    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:12.314020 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:12.313987    4434 scope.go:122] "RemoveContainer" containerID="76b213754c3f432eb01da3c3219e25eb7562f6904e7b0bea58f732f2374c3785"
Nov 04 01:42:17.592654 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:17.592620    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:18.304648 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:18.304614    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:18.572861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:18.572779    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:21.896551 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:42:21.896514    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:42:22.739298 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:22.739269    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:22.919246 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:22.919210    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:23.212647 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:23.212613    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:25.954485 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:25.954448    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:26.178035 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:26.178002    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:28.728582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:28.728539    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:28.999538 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:28.999456    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:29.164962 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:29.164930    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:33.282239 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:33.282206    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:37.879003 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:37.878972    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:42.285117 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:42.285087    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:47.138345 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:47.138307    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:47.644754 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:47.644716    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:48.242595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:48.242538    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:48.421331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:48.421300    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:51.581960 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:51.581927    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:51.804902 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:51.804862    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:42:51.897009 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:42:51.896971    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:42:56.177876 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:42:56.177799    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:00.827626 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:00.827585    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:05.294132 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:05.294087    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:09.751683 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:09.751586    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:11.184015 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:11.183979    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:11.346791 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:11.346757    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:14.623059 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:14.623027    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:19.081197 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:19.081158    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:21.896877 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:43:21.896837    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:43:23.577144 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:23.577110    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:27.996331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:27.996293    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:32.474019 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:32.473984    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:36.864531 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:36.864490    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:41.428983 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:41.428948    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:45.885858 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:45.885820    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:50.368913 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:50.368870    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:51.896532 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:43:51.896497    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:43:52.433810 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:52.433779    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:54.997835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:54.997798    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:57.458200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:57.458166    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:43:59.674644 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:43:59.674611    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:04.103995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:04.103958    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:04.777785 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:04.777752    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:04.936190 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:04.936095    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:08.033193 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:08.033154    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:09.445767 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:09.445735    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:09.792566 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:09.792478    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:10.478258 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:10.478213    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:11.515915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:11.515884    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:16.146892 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:16.146852    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:16.312780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:16.312742    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:16.483277 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:16.483184    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:21.896764 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:44:21.896719    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:44:34.516839 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:34.516758    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:34.695746 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:34.695709    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:40.255610 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:44:40.255571    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:44:51.899506 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:44:51.899468    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:45:06.695502 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:45:06.695465    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:11.854595 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:45:11.854564    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:45:21.896654 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:45:21.896616    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:45:51.902710 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:45:51.902652    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:46:12.406968 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:12.406905    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:12.583915 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:12.583884    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:21.896944 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:46:21.896903    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:46:48.171031 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:48.170994    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:48.842264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:48.842228    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:49.035369 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:49.035333    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:49.868692 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:49.868648    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:51.897272 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:46:51.897225    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:46:52.698119 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:52.698079    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:53.081563 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:53.081474    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:55.968154 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:55.968115    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:56.160189 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:56.160155    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:58.519970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:58.519937    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:58.695823 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:58.695773    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:46:58.890355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:46:58.890317    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:03.216282 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:03.216239    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:07.456787 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:07.456748    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:07.999372 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:07.999326    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:12.509975 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:12.509936    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:13.931489 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:13.931453    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:17.324046 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:17.324010    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:17.492472 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:17.492445    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:18.109659 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:18.109625    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:18.284412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:18.284370    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:20.968422 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:20.968379    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:21.897243 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:47:21.897204    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:47:22.037769 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:22.037727    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:26.476207 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:26.476114    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:30.915433 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:30.915400    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:35.485957 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:35.485925    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:39.896951 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:39.896910    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:44.518170 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:44.518133    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:49.005088 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:49.005046    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:49.686650 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:49.686614    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:51.896539 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:47:51.896498    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:47:53.468110 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:53.468076    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:47:57.986231 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:47:57.986193    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:02.700290 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:02.700252    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:07.193649 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:07.193612    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:12.172417 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:12.172374    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:12.912474 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:12.912437    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:16.573389 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:16.573361    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:17.522040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:17.522005    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:21.458436 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:21.458398    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:21.896007 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:48:21.895970    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:48:22.649061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:22.649028    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:26.000677 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:26.000630    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:29.858752 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:29.858721    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:30.055852 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:30.055822    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:33.977508 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:33.977476    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:34.229971 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:34.229893    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:35.023896 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:35.023860    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:41.363575 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:41.363536    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:41.827002 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:41.826969    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:42.247352 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:42.247321    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:49.015438 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:49.015388    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:48:51.896595 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:48:51.896556    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:48:59.439205 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:48:59.439121    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:00.245153 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:49:00.245111    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:13.253604 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:49:13.253570    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:13.471968 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:49:13.471930    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:15.097600 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:49:15.097565    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:21.895615 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:49:21.895579    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:49:33.557093 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:49:33.557060    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:49:51.896957 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:49:51.896920    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:50:09.366200 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:50:09.366155    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:13.607194 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:50:13.607154    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:21.895982 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:50:21.895939    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:50:23.492908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:50:23.492828    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:50:51.896037 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:50:51.896002    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:50:58.623861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:50:58.623822    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:13.359318 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.359282    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"]
Nov 04 01:51:13.359833 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.359813    4434 cpu_manager.go:414] "containerMap: removing container" podUID="74148ccf-c040-4c5c-9617-85d34cd255b6" containerName="revoke-keys"
Nov 04 01:51:13.359913 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.359835    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74148ccf-c040-4c5c-9617-85d34cd255b6" containerName="revoke-keys"
Nov 04 01:51:13.359974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.359929    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="74148ccf-c040-4c5c-9617-85d34cd255b6" containerName="revoke-keys"
Nov 04 01:51:13.363214 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.363192    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.468636 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.468577    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8zmjb\""
Nov 04 01:51:13.468636 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.468629    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 04 01:51:13.468903 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.468791    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 04 01:51:13.468903 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.468893    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 04 01:51:13.469019 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.468956    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 04 01:51:13.476556 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.476518    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.476742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.476600    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.476742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.476638    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.476742 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.476725    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj4pn\" (UniqueName: \"kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.488784 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.488750    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"]
Nov 04 01:51:13.577519 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.577470    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj4pn\" (UniqueName: \"kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.577731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.577588    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.577731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.577647    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.577731 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.577722    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.578346 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.578319    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.580219 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.580194    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.580316 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.580194    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.634263 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.634236    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj4pn\" (UniqueName: \"kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn\") pod \"maas-api-7894895fc6-tnprv\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") " pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.675320 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.675292    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:13.951814 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:13.951741    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:14.050582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:14.050544    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"]
Nov 04 01:51:14.054068 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:51:14.054037    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00cea5a0_d010_4ae4_b05e_3b8f9feeec80.slice/crio-0e02d811bfbf5dec3bdcc65818e01b7a1b9038bd68ec5bd7af671b13c946ff2a WatchSource:0}: Error finding container 0e02d811bfbf5dec3bdcc65818e01b7a1b9038bd68ec5bd7af671b13c946ff2a: Status 404 returned error can't find the container with id 0e02d811bfbf5dec3bdcc65818e01b7a1b9038bd68ec5bd7af671b13c946ff2a
Nov 04 01:51:14.055279 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:14.055263    4434 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 04 01:51:14.325094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:14.325005    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv" event={"ID":"00cea5a0-d010-4ae4-b05e-3b8f9feeec80","Type":"ContainerStarted","Data":"0e02d811bfbf5dec3bdcc65818e01b7a1b9038bd68ec5bd7af671b13c946ff2a"}
Nov 04 01:51:15.330995 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:15.330911    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv" event={"ID":"00cea5a0-d010-4ae4-b05e-3b8f9feeec80","Type":"ContainerStarted","Data":"4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96"}
Nov 04 01:51:15.331365 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:15.331042    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:15.447634 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:15.447581    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv" podStartSLOduration=1.455914435 podStartE2EDuration="2.447567598s" totalImagesPullingTime="991.653163ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:15.446518102 +0000 UTC m=+3244.108658164" watchObservedRunningTime="2026-11-04 01:51:15.447567598 +0000 UTC m=+3244.109707658"
Nov 04 01:51:20.110487 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:20.110450    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:20.736835 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:20.736802    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:20.917344 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:20.917306    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:21.341035 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.340999    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:21.341544 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.341510    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"]
Nov 04 01:51:21.358181 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.358134    4434 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv" podUID="00cea5a0-d010-4ae4-b05e-3b8f9feeec80" containerName="maas-api" containerID="cri-o://4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96" gracePeriod=30
Nov 04 01:51:21.391355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.391320    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"]
Nov 04 01:51:21.394868 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.394846    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.447974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.447944    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"]
Nov 04 01:51:21.555107 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.555072    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df7799e-cbbd-4477-9e9d-d6aab2935e33-openshift-service-ca\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.555274 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.555234    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skrlg\" (UniqueName: \"kubernetes.io/projected/3df7799e-cbbd-4477-9e9d-d6aab2935e33-kube-api-access-skrlg\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.555375 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.555298    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df7799e-cbbd-4477-9e9d-d6aab2935e33-maas-api-tls\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.555444 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.555416    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df7799e-cbbd-4477-9e9d-d6aab2935e33-metrics-certs\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.609165 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.609136    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:21.656815 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.656772    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df7799e-cbbd-4477-9e9d-d6aab2935e33-maas-api-tls\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.656974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.656829    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df7799e-cbbd-4477-9e9d-d6aab2935e33-metrics-certs\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.656974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.656879    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df7799e-cbbd-4477-9e9d-d6aab2935e33-openshift-service-ca\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.656974 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.656953    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skrlg\" (UniqueName: \"kubernetes.io/projected/3df7799e-cbbd-4477-9e9d-d6aab2935e33-kube-api-access-skrlg\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.657593 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.657570    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3df7799e-cbbd-4477-9e9d-d6aab2935e33-openshift-service-ca\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.659306 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.659275    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3df7799e-cbbd-4477-9e9d-d6aab2935e33-maas-api-tls\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.659426 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.659330    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3df7799e-cbbd-4477-9e9d-d6aab2935e33-metrics-certs\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.714503 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.714465    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skrlg\" (UniqueName: \"kubernetes.io/projected/3df7799e-cbbd-4477-9e9d-d6aab2935e33-kube-api-access-skrlg\") pod \"maas-api-6fc6d86564-jgv98\" (UID: \"3df7799e-cbbd-4477-9e9d-d6aab2935e33\") " pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.743371 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.743337    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:21.757596 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.757570    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs\") pod \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") "
Nov 04 01:51:21.757717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.757617    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls\") pod \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") "
Nov 04 01:51:21.757717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.757656    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn\" (UniqueName: \"kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn\") pod \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") "
Nov 04 01:51:21.757717 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.757709    4434 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca\") pod \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\" (UID: \"00cea5a0-d010-4ae4-b05e-3b8f9feeec80\") "
Nov 04 01:51:21.758106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.758081    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca" pod "00cea5a0-d010-4ae4-b05e-3b8f9feeec80" (UID: "00cea5a0-d010-4ae4-b05e-3b8f9feeec80"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 04 01:51:21.759857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.759820    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls" pod "00cea5a0-d010-4ae4-b05e-3b8f9feeec80" (UID: "00cea5a0-d010-4ae4-b05e-3b8f9feeec80"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:51:21.759857 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.759839    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs" pod "00cea5a0-d010-4ae4-b05e-3b8f9feeec80" (UID: "00cea5a0-d010-4ae4-b05e-3b8f9feeec80"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 04 01:51:21.759984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.759950    4434 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn" pod "00cea5a0-d010-4ae4-b05e-3b8f9feeec80" (UID: "00cea5a0-d010-4ae4-b05e-3b8f9feeec80"). InnerVolumeSpecName "kube-api-access-pj4pn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 04 01:51:21.859087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.859004    4434 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-maas-api-tls\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:51:21.859087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.859034    4434 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj4pn\" (UniqueName: \"kubernetes.io/projected/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-kube-api-access-pj4pn\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:51:21.859087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.859044    4434 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-openshift-service-ca\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:51:21.859087 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:21.859052    4434 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00cea5a0-d010-4ae4-b05e-3b8f9feeec80-metrics-certs\") on node \"ip-10-0-136-128.us-west-2.compute.internal\" DevicePath \"\""
Nov 04 01:51:21.900244 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:51:21.900201    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:51:22.117479 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.117397    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"]
Nov 04 01:51:22.121072 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:51:22.121037    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df7799e_cbbd_4477_9e9d_d6aab2935e33.slice/crio-57dc5c34719d69df06dccdd2f4b0f22d3fcca630fefc4a37c466665877d318a9 WatchSource:0}: Error finding container 57dc5c34719d69df06dccdd2f4b0f22d3fcca630fefc4a37c466665877d318a9: Status 404 returned error can't find the container with id 57dc5c34719d69df06dccdd2f4b0f22d3fcca630fefc4a37c466665877d318a9
Nov 04 01:51:22.364355 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.364319    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98" event={"ID":"3df7799e-cbbd-4477-9e9d-d6aab2935e33","Type":"ContainerStarted","Data":"57dc5c34719d69df06dccdd2f4b0f22d3fcca630fefc4a37c466665877d318a9"}
Nov 04 01:51:22.365562 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.365534    4434 generic.go:356] "Generic (PLEG): container finished" podID="00cea5a0-d010-4ae4-b05e-3b8f9feeec80" containerID="4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96" exitCode=0
Nov 04 01:51:22.365682 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.365578    4434 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"
Nov 04 01:51:22.365682 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.365618    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv" event={"ID":"00cea5a0-d010-4ae4-b05e-3b8f9feeec80","Type":"ContainerDied","Data":"4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96"}
Nov 04 01:51:22.365774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.365685    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7894895fc6-tnprv" event={"ID":"00cea5a0-d010-4ae4-b05e-3b8f9feeec80","Type":"ContainerDied","Data":"0e02d811bfbf5dec3bdcc65818e01b7a1b9038bd68ec5bd7af671b13c946ff2a"}
Nov 04 01:51:22.365774 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.365711    4434 scope.go:122] "RemoveContainer" containerID="4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96"
Nov 04 01:51:22.384010 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.383989    4434 scope.go:122] "RemoveContainer" containerID="4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96"
Nov 04 01:51:22.384325 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:51:22.384305    4434 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96\": container with ID starting with 4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96 not found: ID does not exist" containerID="4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96"
Nov 04 01:51:22.384378 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.384334    4434 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96"} err="failed to get container status \"4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96\": rpc error: code = NotFound desc = could not find container \"4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96\": container with ID starting with 4da1bb88f09975f7bf3fb529b399d5bdf5741a1aa0540aa3d25dec9958fe7b96 not found: ID does not exist"
Nov 04 01:51:22.530085 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.530047    4434 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"]
Nov 04 01:51:22.534050 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:22.534014    4434 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7894895fc6-tnprv"]
Nov 04 01:51:23.372918 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:23.372877    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98" event={"ID":"3df7799e-cbbd-4477-9e9d-d6aab2935e33","Type":"ContainerStarted","Data":"78d84e346de6926501f82a8e79aa0ead749bef63dea24211888c634308e9c38d"}
Nov 04 01:51:23.373334 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:23.372951    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:23.473972 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:23.473940    4434 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00cea5a0-d010-4ae4-b05e-3b8f9feeec80" path="/var/lib/kubelet/pods/00cea5a0-d010-4ae4-b05e-3b8f9feeec80/volumes"
Nov 04 01:51:23.490254 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:23.490199    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98" podStartSLOduration=1.574837372 podStartE2EDuration="2.490184131s" totalImagesPullingTime="915.346759ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:51:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:51:23.489472498 +0000 UTC m=+3252.151612571" watchObservedRunningTime="2026-11-04 01:51:23.490184131 +0000 UTC m=+3252.152324186"
Nov 04 01:51:23.902997 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:23.902955    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:24.098553 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:24.098516    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:27.100979 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:27.100943    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:27.320937 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:27.320902    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:27.870000 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:27.869966    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:28.032026 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:28.031989    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:29.384395 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:29.384367    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6fc6d86564-jgv98"
Nov 04 01:51:32.711374 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:32.711340    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:37.288264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:37.288229    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:41.766787 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:41.766743    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:46.570624 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:46.570590    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:51.170874 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:51.170843    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:51:51.895875 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:51:51.895838    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:51:55.673362 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:51:55.673289    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:00.197589 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:00.197551    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:04.577549 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:04.577514    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:07.089860 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:07.089824    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:07.261564 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:07.261535    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:08.971523 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:08.971484    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:12.882569 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:12.882527    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:14.265190 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:14.265155    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:15.780061 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:15.780024    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:16.322042 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:16.322008    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:17.613794 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:17.613753    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:18.237567 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:18.237530    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:18.467716 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:18.467655    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:21.897009 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:52:21.896972    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:52:22.870970 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:22.870934    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:23.049723 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:23.049658    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:23.219797 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:23.219763    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:26.081251 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:26.081218    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:26.259830 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:26.259799    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.532797 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:28.532758    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.732529 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:28.732475    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:28.903166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:28.903136    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:33.068163 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:33.068130    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:37.604427 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:37.604373    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:41.987840 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:41.987802    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:42.576843 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:42.576813    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:46.782173 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:46.782140    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:51.275412 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:51.275381    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:52:51.896926 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:52:51.896884    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:52:55.758985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:52:55.758922    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:00.180601 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:00.180560    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:04.697227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:04.697188    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:09.085049 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:09.085012    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:13.608854 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:13.608811    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:14.626260 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:14.626228    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:17.965106 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:17.965059    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:21.896828 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:53:21.896795    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:53:22.450101 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:22.450068    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:26.907266 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:26.907185    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:31.374413 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:31.374376    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:35.771007 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:35.770968    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:40.269509 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:40.269477    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:44.668138 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:44.668088    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:49.185097 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:49.185061    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:51.897040 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:53:51.897003    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:53:53.570948 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:53.570914    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:53:58.090899 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:53:58.090868    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:00.822632 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:00.822596    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:00.991986 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:00.991953    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:02.570900 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:02.570868    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:07.071369 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:07.071335    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:11.625166 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:11.625137    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:14.974227 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:14.974191    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:16.339528 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:16.339490    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:21.011331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:21.011294    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:21.897240 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:54:21.897194    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:54:25.496195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:25.496161    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:25.686399 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:25.686360    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:33.683331 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:33.683295    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:35.074628 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:35.074596    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:35.338908 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:35.338811    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:36.093980 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:36.093942    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:41.961033 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:41.960993    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:42.211723 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:42.211631    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:42.478030 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:42.477930    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:46.222420 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:54:46.222385    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:54:51.896204 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:54:51.896165    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:55:15.288066 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:55:15.288029    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:55:21.896612 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:55:21.896570    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:55:51.896502 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:55:51.896455    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:56:09.165533 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:09.165499    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-k8tl7_3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c/manager/0.log"
Nov 04 01:56:11.679875 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:11.679846    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6fc6d86564-jgv98_3df7799e-cbbd-4477-9e9d-d6aab2935e33/maas-api/0.log"
Nov 04 01:56:15.137045 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:15.137016    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rhn29_94adf17c-1b38-47b9-b02b-a668f858d8a7/kuadrant-console-plugin/0.log"
Nov 04 01:56:15.359616 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:15.359580    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:15.536335 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:15.536251    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:15.813684 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:15.813582    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:17.509904 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:17.509876    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5794fb467c-wv6mv_64ae051b-00f6-4d73-b124-5677d654f54e/kube-auth-proxy/0.log"
Nov 04 01:56:17.741984 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:17.741953    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-2sl8p_179fdcce-5744-45d1-ad03-437346736c4d/istio-proxy/0.log"
Nov 04 01:56:18.016142 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:18.016104    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-4q2t9_464e35a4-a96d-4e48-a38a-9ef537538445/istio-proxy/0.log"
Nov 04 01:56:18.973307 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:18.973275    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:19.148788 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:19.148734    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:19.272108 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:19.272015    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-j72g2_83e28f11-e50b-4523-b8a5-4647d841c30c/istio-proxy/0.log"
Nov 04 01:56:21.516808 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:21.516736    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:56:21.895934 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:56:21.895900    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:56:34.689195 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:34.689161    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-k8tl7_3bd0cca4-6d58-4ea9-8d2b-b9df30eefc8c/manager/0.log"
Nov 04 01:56:36.673779 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:36.673742    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6fc6d86564-jgv98_3df7799e-cbbd-4477-9e9d-d6aab2935e33/maas-api/0.log"
Nov 04 01:56:39.782028 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:39.781949    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rhn29_94adf17c-1b38-47b9-b02b-a668f858d8a7/kuadrant-console-plugin/0.log"
Nov 04 01:56:41.841324 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:41.841294    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5794fb467c-wv6mv_64ae051b-00f6-4d73-b124-5677d654f54e/kube-auth-proxy/0.log"
Nov 04 01:56:42.068040 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:42.068012    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-2sl8p_179fdcce-5744-45d1-ad03-437346736c4d/istio-proxy/0.log"
Nov 04 01:56:42.357157 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:42.357108    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-4q2t9_464e35a4-a96d-4e48-a38a-9ef537538445/istio-proxy/0.log"
Nov 04 01:56:43.642799 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:56:43.642744    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-694db55c87-j72g2_83e28f11-e50b-4523-b8a5-4647d841c30c/istio-proxy/0.log"
Nov 04 01:56:51.895535 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:56:51.895497    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:57:01.817469 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:57:01.817431    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:15.958005 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:57:15.957953    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:21.896153 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:57:21.896103    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:57:51.895909 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:57:51.895874    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:57:58.918930 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:57:58.918850    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:57:59.150078 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:57:59.150042    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:11.598914 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:11.598873    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rcwm8_d41414c4-11d2-4e45-aa39-8f33726a40fd/global-pull-secret-syncer/0.log"
Nov 04 01:58:13.635240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:13.635208    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log"
Nov 04 01:58:15.613873 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:15.613831    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-128.us-west-2.compute.internal_1d4b0eb194eb22fd85d7942f84eead7e/haproxy/0.log"
Nov 04 01:58:16.290713 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:16.290649    4434 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-456n2_de508010-fa70-4f37-a691-bdecaecdd3e7/konnectivity-agent/0.log" limit=1024
Nov 04 01:58:19.435367 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.435338    4434 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"]
Nov 04 01:58:19.435780 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.435765    4434 cpu_manager.go:414] "containerMap: removing container" podUID="00cea5a0-d010-4ae4-b05e-3b8f9feeec80" containerName="maas-api"
Nov 04 01:58:19.435828 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.435781    4434 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00cea5a0-d010-4ae4-b05e-3b8f9feeec80" containerName="maas-api"
Nov 04 01:58:19.435881 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.435871    4434 memory_manager.go:365] "RemoveStaleState removing state" podUID="00cea5a0-d010-4ae4-b05e-3b8f9feeec80" containerName="maas-api"
Nov 04 01:58:19.438847 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.438832    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.492256 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.492221    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7gzq6\"/\"kube-root-ca.crt\""
Nov 04 01:58:19.492391 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.492292    4434 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7gzq6\"/\"openshift-service-ca.crt\""
Nov 04 01:58:19.526778 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.526747    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-proc\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.526897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.526788    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-podres\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.526897 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.526820    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zztxj\" (UniqueName: \"kubernetes.io/projected/91815446-53e1-4591-9d52-4eec7041a5e2-kube-api-access-zztxj\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.527007 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.526979    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-sys\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.527094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.527082    4434 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-lib-modules\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.544179 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.544155    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"]
Nov 04 01:58:19.592817 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.592787    4434 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7gzq6\"/\"default-dockercfg-pd6hx\""
Nov 04 01:58:19.627863 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.627834    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-podres\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.627998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.627884    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zztxj\" (UniqueName: \"kubernetes.io/projected/91815446-53e1-4591-9d52-4eec7041a5e2-kube-api-access-zztxj\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.627998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.627923    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-sys\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.627998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.627957    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-podres\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.627998 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.627967    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-lib-modules\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.628235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.628018    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-lib-modules\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.628235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.628024    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-sys\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.628235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.628035    4434 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-proc\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.628235 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.628070    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/91815446-53e1-4591-9d52-4eec7041a5e2-proc\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.685996 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.685930    4434 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zztxj\" (UniqueName: \"kubernetes.io/projected/91815446-53e1-4591-9d52-4eec7041a5e2-kube-api-access-zztxj\") pod \"perf-node-gather-daemonset-4gfjm\" (UID: \"91815446-53e1-4591-9d52-4eec7041a5e2\") " pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:19.751264 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:19.751238    4434 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:20.122240 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:20.122164    4434 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"]
Nov 04 01:58:20.129106 ip-10-0-136-128 kubenswrapper[4434]: W1104 01:58:20.128731    4434 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod91815446_53e1_4591_9d52_4eec7041a5e2.slice/crio-86dfcb618e219f336f615af4483566cc80e8cdc6b46c2175982ab5c2dbc38947 WatchSource:0}: Error finding container 86dfcb618e219f336f615af4483566cc80e8cdc6b46c2175982ab5c2dbc38947: Status 404 returned error can't find the container with id 86dfcb618e219f336f615af4483566cc80e8cdc6b46c2175982ab5c2dbc38947
Nov 04 01:58:20.262043 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:20.262006    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm" event={"ID":"91815446-53e1-4591-9d52-4eec7041a5e2","Type":"ContainerStarted","Data":"50efa54cc9d9c46aceb572bb8b35fad8f7db9b03a8dcd9abaeca4ac04426eb10"}
Nov 04 01:58:20.262201 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:20.262067    4434 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm" event={"ID":"91815446-53e1-4591-9d52-4eec7041a5e2","Type":"ContainerStarted","Data":"86dfcb618e219f336f615af4483566cc80e8cdc6b46c2175982ab5c2dbc38947"}
Nov 04 01:58:20.262201 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:20.262091    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:20.387824 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:20.387780    4434 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm" podStartSLOduration=1.387767161 podStartE2EDuration="1.387767161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:58:20.386619248 +0000 UTC m=+3669.048759308" watchObservedRunningTime="2026-11-04 01:58:20.387767161 +0000 UTC m=+3669.049907221"
Nov 04 01:58:20.409313 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:20.409292    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-rhn29_94adf17c-1b38-47b9-b02b-a668f858d8a7/kuadrant-console-plugin/0.log"
Nov 04 01:58:21.896023 ip-10-0-136-128 kubenswrapper[4434]: E1104 01:58:21.895986    4434 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-26d2d" containerName="extractor"
Nov 04 01:58:24.631026 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:24.631000    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/alertmanager/0.log"
Nov 04 01:58:25.069582 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:25.069511    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/config-reloader/0.log"
Nov 04 01:58:25.508985 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:25.508958    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/kube-rbac-proxy-web/0.log"
Nov 04 01:58:25.947094 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:25.947066    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/kube-rbac-proxy/0.log"
Nov 04 01:58:26.274920 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:26.274850    4434 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7gzq6/perf-node-gather-daemonset-4gfjm"
Nov 04 01:58:26.393037 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:26.392995    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/kube-rbac-proxy-metric/0.log"
Nov 04 01:58:26.831753 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:26.831724    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/prom-label-proxy/0.log"
Nov 04 01:58:27.270002 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:27.269975    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_38b8e65b-f464-4e11-9db7-756974868b61/init-config-reloader/0.log"
Nov 04 01:58:28.259846 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:28.259809    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-5ntrn_4ccbf275-c8e0-40d2-923a-699910bc0c53/kube-state-metrics/0.log"
Nov 04 01:58:28.726464 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:28.726433    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-5ntrn_4ccbf275-c8e0-40d2-923a-699910bc0c53/kube-rbac-proxy-main/0.log"
Nov 04 01:58:29.165599 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:29.165566    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-5ntrn_4ccbf275-c8e0-40d2-923a-699910bc0c53/kube-rbac-proxy-self/0.log"
Nov 04 01:58:29.613069 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:29.613041    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-9958fc47b-vts5v_cb9f1f13-14ee-4f50-80ce-dc8ada58f953/metrics-server/0.log"
Nov 04 01:58:31.851989 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:31.851961    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m4jls_ed2a3230-bc57-4fe5-8a85-1487aab119c3/node-exporter/0.log"
Nov 04 01:58:32.288879 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:32.288849    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m4jls_ed2a3230-bc57-4fe5-8a85-1487aab119c3/kube-rbac-proxy/0.log"
Nov 04 01:58:32.729396 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:32.729368    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m4jls_ed2a3230-bc57-4fe5-8a85-1487aab119c3/init-textfile/0.log"
Nov 04 01:58:34.508356 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:34.508326    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-fzthg_03166a27-52e3-4e11-aeeb-d6d851cd42c4/kube-rbac-proxy-main/0.log"
Nov 04 01:58:35.013917 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:35.013886    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-fzthg_03166a27-52e3-4e11-aeeb-d6d851cd42c4/kube-rbac-proxy-self/0.log"
Nov 04 01:58:35.453866 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:35.453839    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-fzthg_03166a27-52e3-4e11-aeeb-d6d851cd42c4/openshift-state-metrics/0.log"
Nov 04 01:58:35.904576 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:35.904547    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/prometheus/0.log"
Nov 04 01:58:36.341403 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:36.341332    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/config-reloader/0.log"
Nov 04 01:58:36.783993 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:36.783960    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/thanos-sidecar/0.log"
Nov 04 01:58:37.220788 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:37.220759    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/kube-rbac-proxy-web/0.log"
Nov 04 01:58:37.658700 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:37.658647    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/kube-rbac-proxy/0.log"
Nov 04 01:58:38.097803 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:38.097737    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/kube-rbac-proxy-thanos/0.log"
Nov 04 01:58:38.537048 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:38.537016    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7e75b897-7a6b-45d1-bfbd-2bc050c490d1/init-config-reloader/0.log"
Nov 04 01:58:39.872222 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:39.872169    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-xxspl_11cbfdab-08e2-45a8-a2c6-f0d37a871563/prometheus-operator-admission-webhook/0.log"
Nov 04 01:58:40.320861 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:40.320828    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-797775ffd6-zlbr7_5f4cc4c0-41fa-45cd-ae33-60570b10b12e/telemeter-client/0.log"
Nov 04 01:58:40.763447 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:40.763412    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-797775ffd6-zlbr7_5f4cc4c0-41fa-45cd-ae33-60570b10b12e/reload/0.log"
Nov 04 01:58:41.205734 ip-10-0-136-128 kubenswrapper[4434]: I1104 01:58:41.205687    4434 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-797775ffd6-zlbr7_5f4cc4c0-41fa-45cd-ae33-60570b10b12e/kube-rbac-proxy/0.log"