Oct 31 03:25:26.620427 ip-10-0-129-108 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 31 03:25:26.620442 ip-10-0-129-108 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 31 03:25:26.620450 ip-10-0-129-108 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 31 03:25:26.620802 ip-10-0-129-108 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 31 03:25:36.666385 ip-10-0-129-108 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 31 03:25:36.666399 ip-10-0-129-108 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8b726ae01adf4b81aba16e86d6ebb978 --
Oct 31 03:27:55.964550 ip-10-0-129-108 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 03:27:56.353312 ip-10-0-129-108 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:27:56.353312 ip-10-0-129-108 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 03:27:56.353312 ip-10-0-129-108 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:27:56.353312 ip-10-0-129-108 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:27:56.356694 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356603    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.356694 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356694    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.356694 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356698    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356701    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356705    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356708    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356710    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356714    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356716    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356719    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356722    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356725    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356728    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356731    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356733    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356736    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356738    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356741    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356744    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356746    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356749    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356751    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.356790 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356754    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356756    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356759    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356761    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356764    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356766    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356777    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356779    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356782    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356785    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356789    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356793    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356797    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356801    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356804    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356807    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356810    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356812    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356815    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.357271 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356817    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356820    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356823    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356825    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356828    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356831    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356833    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356836    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356839    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356841    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356844    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356846    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356849    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356851    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356854    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356857    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356860    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356863    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356865    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356867    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.357719 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356870    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356872    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356875    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356878    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356880    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356883    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356885    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356888    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356890    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356893    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356896    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356899    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356902    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356904    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356907    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356910    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356913    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356916    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356918    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356921    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.358194 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356925    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356928    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356930    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356933    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356937    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356940    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356943    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356947    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356950    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356953    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356955    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356958    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356962    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356965    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356967    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356970    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356973    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356975    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356978    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.358681 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356980    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.359119 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356983    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.359119 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356985    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.359119 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356988    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.359119 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356990    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.359119 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.356993    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.359549 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359528    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.359549 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359548    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359553    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359557    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359560    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359563    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359565    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359569    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359572    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359574    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359578    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359583    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359586    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359589    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359591    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359594    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359597    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359600    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359602    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359605    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359608    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.359615 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359610    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359613    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359616    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359619    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359622    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359625    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359628    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359631    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359633    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359636    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359639    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359642    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359645    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359648    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359650    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359653    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359655    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359658    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359660    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359663    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.360196 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359665    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359668    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359670    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359672    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359675    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359677    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359680    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359682    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359685    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359687    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359690    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359693    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359696    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359699    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359701    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359704    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359707    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359710    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359712    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.360699 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359715    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359718    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359720    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359723    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359725    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359728    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359731    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359734    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359737    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359740    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359742    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359745    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359748    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359750    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359753    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359755    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359758    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359760    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359763    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359765    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.361162 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359768    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359771    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359773    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359776    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359781    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359791    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359797    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359800    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359803    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359806    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359809    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359811    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359814    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359817    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359819    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359822    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359825    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359827    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.361672 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359830    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359833    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359836    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359839    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359842    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359845    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359847    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.359850    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360391    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360397    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360400    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360403    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360406    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360409    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360411    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360414    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360416    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360419    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360423    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.362094 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360425    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360429    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360432    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360435    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360438    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360440    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360443    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360446    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360448    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360451    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360453    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360456    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360458    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360461    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360464    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360467    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360470    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360473    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360475    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360478    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.362570 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360480    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360483    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360486    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360488    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360491    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360493    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360495    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360498    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360501    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360503    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360506    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360509    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360512    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360515    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360518    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360521    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360523    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360526    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360528    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.363069 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360531    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360534    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360536    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360539    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360541    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360543    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360546    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360548    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360553    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360558    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360561    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360564    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360568    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360571    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360573    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360576    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360579    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360581    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360585    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.363541 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360588    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360591    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360594    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360597    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360599    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360602    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360605    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360608    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360611    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360614    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360616    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360619    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360621    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360624    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360626    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360629    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360631    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360634    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360636    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360639    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.364003 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360641    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360644    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360647    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360649    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360652    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360654    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360657    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360660    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360662    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360665    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360667    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360670    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360673    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360676    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360678    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360681    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.360684    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361738    2583 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361749    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361759    2583 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361764    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 03:27:56.364515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361769    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361772    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361777    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361782    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361785    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361789    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361793    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361796    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361799    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361803    2583 flags.go:64] FLAG: --cgroup-root=""
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361805    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361808    2583 flags.go:64] FLAG: --client-ca-file=""
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361811    2583 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361814    2583 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361819    2583 flags.go:64] FLAG: --cluster-domain=""
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361822    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361825    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361829    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361832    2583 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361836    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361839    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361843    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361847    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361850    2583 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 03:27:56.365012 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361853    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361856    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361859    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361863    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361868    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361871    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361875    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361878    2583 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361881    2583 flags.go:64] FLAG: --enable-server="true"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361884    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361894    2583 flags.go:64] FLAG: --event-burst="100"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361898    2583 flags.go:64] FLAG: --event-qps="50"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361901    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361905    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361908    2583 flags.go:64] FLAG: --eviction-hard=""
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361912    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361915    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361919    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361922    2583 flags.go:64] FLAG: --eviction-soft=""
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361925    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361928    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361931    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361934    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361938    2583 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361941    2583 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 03:27:56.365583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361944    2583 flags.go:64] FLAG: --feature-gates=""
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361948    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361952    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361955    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361958    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361962    2583 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361965    2583 flags.go:64] FLAG: --help="false"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361968    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361971    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361974    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361977    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361981    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361984    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361987    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361990    2583 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361993    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361996    2583 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.361999    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362003    2583 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362006    2583 flags.go:64] FLAG: --kube-reserved=""
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362009    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362011    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362014    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 03:27:56.366197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362017    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362020    2583 flags.go:64] FLAG: --lock-file=""
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362023    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362026    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362029    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362035    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362038    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362041    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362045    2583 flags.go:64] FLAG: --logging-format="text"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362048    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362051    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362054    2583 flags.go:64] FLAG: --manifest-url=""
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362057    2583 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362062    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362066    2583 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362070    2583 flags.go:64] FLAG: --max-pods="110"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362073    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362075    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362078    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362082    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362085    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362088    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362092    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362101    2583 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362104    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 03:27:56.366758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362108    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362111    2583 flags.go:64] FLAG: --pod-cidr=""
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362114    2583 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362117    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362120    2583 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362123    2583 flags.go:64] FLAG: --port="10250"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362126    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362129    2583 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-04ba6f89ae360b258"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362132    2583 flags.go:64] FLAG: --qos-reserved=""
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362136    2583 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362139    2583 flags.go:64] FLAG: --register-node="true"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362142    2583 flags.go:64] FLAG: --register-with-taints=""
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362146    2583 flags.go:64] FLAG: --registry-burst="10"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362149    2583 flags.go:64] FLAG: --registry-qps="5"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362152    2583 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362156    2583 flags.go:64] FLAG: --reserved-memory=""
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362160    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362163    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362166    2583 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362170    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362173    2583 flags.go:64] FLAG: --runonce="false"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362176    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362179    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362183    2583 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362186    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362189    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 03:27:56.367404 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362192    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362196    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362199    2583 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362212    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362216    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362220    2583 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362223    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362226    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362229    2583 flags.go:64] FLAG: --system-cgroups=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362232    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362238    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362241    2583 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362244    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362248    2583 flags.go:64] FLAG: --tls-min-version=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362251    2583 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362254    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362257    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362260    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362263    2583 flags.go:64] FLAG: --v="2"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362271    2583 flags.go:64] FLAG: --version="false"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362275    2583 flags.go:64] FLAG: --vmodule=""
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362279    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362283    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362455    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.368062 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362460    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362463    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362466    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362470    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362473    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362476    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362479    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362482    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362485    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362487    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362490    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362493    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362496    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362498    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362502    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362504    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362507    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362510    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362512    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.368905 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362515    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362518    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362520    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362523    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362525    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362529    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362531    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362534    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362538    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362540    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362543    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362545    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362560    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362565    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362568    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362571    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362576    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362579    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362582    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362586    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.369758 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362588    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362591    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362593    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362596    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362599    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362601    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362604    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362606    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362609    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362612    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362614    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362617    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362620    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362623    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362626    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362629    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362631    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362634    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362636    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362638    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.370618 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362643    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362645    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362648    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362650    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362653    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362656    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362659    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362661    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362665    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362668    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362670    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362673    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362676    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362678    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362681    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362683    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362686    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362688    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362691    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362694    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.371484 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362696    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362699    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362702    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362704    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362707    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362709    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362712    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362714    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362717    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362719    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362722    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362724    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362728    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362731    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362733    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362737    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362741    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362745    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362748    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.372376 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362752    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362756    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362759    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362762    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362764    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362767    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362769    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.362797    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362835    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362839    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362842    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362845    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362848    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362850    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362853    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.373336 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362857    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362861    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362865    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362869    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362872    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362875    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362877    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362880    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362883    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362886    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362890    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362893    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362896    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362899    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362901    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362904    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362907    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362909    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362913    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.373884 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362917    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362920    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362922    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362925    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362927    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362930    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362933    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362935    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362938    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362940    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362943    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362945    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362948    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362950    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362953    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362955    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362958    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362961    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362963    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362965    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.374456 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362968    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362971    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362973    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362977    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362980    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362982    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362985    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362987    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362989    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362992    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362994    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.362997    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363002    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363004    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363007    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363009    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363012    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363014    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363017    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363020    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.375280 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363022    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363025    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363028    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363030    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363034    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363037    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363040    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363042    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363045    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363047    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363050    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363053    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363055    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363058    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363060    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363067    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363070    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363073    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363076    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.375810 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363079    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363081    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363084    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363086    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363089    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363092    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363098    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363101    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363104    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363106    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363109    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363111    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363114    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363117    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363119    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363121    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363124    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363126    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363129    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363131    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.376390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.363134    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.371337    2583 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.371362    2583 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.371387    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.371395    2583 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371492    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371501    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371507    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371511    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371516    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371521    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371525    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371529    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371534    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371539    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371543    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371547    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371552    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371556    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371560    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.377054 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371565    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371569    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371573    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371578    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371583    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371587    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371591    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371595    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371599    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371603    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371609    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371613    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371617    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371622    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371635    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371640    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371644    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371649    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371653    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.377753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371657    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371662    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371667    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371671    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371675    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371680    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371684    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371688    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371693    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371697    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371701    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371707    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371711    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371715    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371719    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371723    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371728    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371732    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371736    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371740    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.378218 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371745    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371749    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371753    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371758    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371762    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371767    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371771    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371776    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371796    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371801    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371805    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371809    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371814    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371818    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371824    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371831    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371836    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371840    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371845    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.379296 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371849    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371854    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371858    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371862    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371867    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371873    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371881    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371886    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371891    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371895    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371900    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371904    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371908    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371913    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371917    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371921    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371925    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371930    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371934    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.379738 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371939    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371943    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371947    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371960    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371965    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371969    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371974    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371978    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371982    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371986    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371991    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371995    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.371999    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372003    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.372056    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372372    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.380222 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372381    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372386    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372390    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372394    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372399    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372403    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372407    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372412    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372416    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372420    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372425    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372429    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372433    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372437    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372441    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372447    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372451    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372456    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372460    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.380627 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372465    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372480    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372487    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372492    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372496    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372500    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372513    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372518    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372522    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372527    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372531    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372535    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372539    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372543    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372547    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372551    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372556    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372560    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372565    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.381165 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372569    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372573    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372577    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372582    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372586    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372590    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372594    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372598    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372603    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372608    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372613    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372617    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372621    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372626    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372631    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372650    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372654    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372658    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372662    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372667    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.381902 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372671    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372675    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372679    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372683    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372688    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372694    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372700    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372705    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372710    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372715    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372720    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372724    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372729    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372733    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372737    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372741    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372746    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372750    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372754    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372758    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.382739 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372763    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372768    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372772    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372776    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372780    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372784    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372788    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372792    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372804    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372809    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372814    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372818    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372822    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372826    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372830    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372834    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372838    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372842    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372847    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372851    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.383321 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372855    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372859    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372864    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372868    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372872    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372876    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372880    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.372920    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372981    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372987    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372991    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372995    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.372999    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373004    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373008    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373013    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:56.383887 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373017    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373021    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373025    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373030    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373034    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373047    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373052    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373056    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373060    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373065    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373069    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373073    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373077    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373081    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373085    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373090    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373094    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373098    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373102    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373107    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:56.384353 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373111    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373115    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373119    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373124    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373128    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373132    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373136    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373140    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373147    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373154    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373159    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373164    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373168    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373173    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373177    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373181    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373186    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373190    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373217    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:56.384921 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373222    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373225    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373229    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373233    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373237    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373241    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373245    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373249    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373253    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373257    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373260    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373264    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373268    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373272    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373276    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373280    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373284    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373288    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373292    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373296    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:56.385409 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373300    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373304    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373310    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373313    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373317    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373321    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373325    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373329    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373333    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373337    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373341    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373345    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373359    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373365    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373369    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373373    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373377    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373382    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373386    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373393    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:56.385903 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373397    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373401    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373406    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373410    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373414    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373418    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373422    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373427    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373432    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373436    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373441    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373445    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373449    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373453    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373458    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373462    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373468    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373472    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:56.373477    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:56.386442 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.373582    2583 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 03:27:56.386892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.377401    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 03:27:56.386892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.378335    2583 server.go:1014] "Starting client certificate rotation"
Oct 31 03:27:56.386892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.378440    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 03:27:56.386892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.378473    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 03:27:56.386892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.382819    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:27:56.387026 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.386987    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:27:56.397087 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.397062    2583 log.go:25] "Validated CRI v1 runtime API"
Oct 31 03:27:56.400476 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.400462    2583 log.go:25] "Validated CRI v1 image API"
Oct 31 03:27:56.401383 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.401361    2583 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 03:27:56.404019 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.403995    2583 fs.go:135] Filesystem UUIDs: map[6d4dc3bf-8bf1-4e56-b097-7cc9ec6dfdb0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 815a233a-6e66-4944-b74b-46e206fc5792:/dev/nvme0n1p3]
Oct 31 03:27:56.404070 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.404020    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 03:27:56.411432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.411274    2583 manager.go:217] Machine: {Timestamp:2026-10-31 03:27:56.408847285 +0000 UTC m=+0.328008520 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102931 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec286c79c75a8890acfa42712a9ec5bc SystemUUID:ec286c79-c75a-8890-acfa-42712a9ec5bc BootID:8b726ae0-1adf-4b81-aba1-6e86d6ebb978 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:cf:27:7f:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:cf:27:7f:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:52:d0:b4:6b:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 03:27:56.411432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.411428    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 03:27:56.411618 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.411602    2583 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 03:27:56.414491 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.414460    2583 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 03:27:56.414637 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.414493    2583 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-108.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 03:27:56.414694 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.414639    2583 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 03:27:56.414694 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.414649    2583 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 03:27:56.414694 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.414680    2583 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:27:56.414767 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.414699    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:27:56.415544 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.415533    2583 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 03:27:56.419506 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.419492    2583 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 03:27:56.419649 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.419640    2583 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 03:27:56.420346 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.420336    2583 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 03:27:56.420403 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.420352    2583 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 03:27:56.420403 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.420366    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 03:27:56.420403 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.420381    2583 kubelet.go:405] "Adding apiserver pod source"
Oct 31 03:27:56.420403 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.420389    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 03:27:56.420812 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.420794    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:27:56.423405 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.423369    2583 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 03:27:56.430390 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.430370    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:27:56.431217 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431194    2583 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 03:27:56.431269 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431237    2583 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 03:27:56.431417 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431403    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 03:27:56.431464 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431425    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 03:27:56.431464 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431433    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 03:27:56.431464 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431447    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 03:27:56.431464 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431457    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 03:27:56.431464 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.431466    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 03:27:56.432139 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432129    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 03:27:56.432139 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432139    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 03:27:56.432231 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432148    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 03:27:56.432231 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432154    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 03:27:56.432231 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432163    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 03:27:56.432231 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432171    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 03:27:56.432853 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432842    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 03:27:56.432899 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.432854    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 03:27:56.436691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.436672    2583 server.go:1263] "Started kubelet"
Oct 31 03:27:56.436843 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.436739    2583 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 03:27:56.437173 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.437115    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 03:27:56.437263 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.437200    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 03:27:56.437529 ip-10-0-129-108 systemd[1]: Started Kubernetes Kubelet.
Oct 31 03:27:56.439233 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.439196    2583 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 03:27:56.439327 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.439294    2583 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 03:27:56.443368 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.443343    2583 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 03:27:56.443948 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.443929    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 03:27:56.444652 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.444634    2583 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 03:27:56.445295 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.445245    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:56.445408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.445390    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 03:27:56.445879 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.445562    2583 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 03:27:56.445879 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.445578    2583 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 03:27:56.445879 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.445683    2583 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 03:27:56.445879 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.445693    2583 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 03:27:56.446220 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446188    2583 factory.go:55] Registering systemd factory
Oct 31 03:27:56.446265 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446225    2583 factory.go:223] Registration of the systemd container factory successfully
Oct 31 03:27:56.446453 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446442    2583 factory.go:153] Registering CRI-O factory
Oct 31 03:27:56.446453 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446452    2583 factory.go:223] Registration of the crio container factory successfully
Oct 31 03:27:56.446530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446500    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 03:27:56.446530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446517    2583 factory.go:103] Registering Raw factory
Oct 31 03:27:56.446530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.446529    2583 manager.go:1199] Started watching for new ooms in manager
Oct 31 03:27:56.447702 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.447497    2583 manager.go:319] Starting recovery of all containers
Oct 31 03:27:56.462466 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.462448    2583 manager.go:324] Recovery completed
Oct 31 03:27:56.467197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.467183    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.469082 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469065    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.469150 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469094    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.469150 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469108    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.469683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469671    2583 cpu_manager.go:226] "Starting" policy="none"
Oct 31 03:27:56.469738 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469683    2583 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 03:27:56.469738 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469686    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.469738 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.469706    2583 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 03:27:56.470410 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.470392    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.470482 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.470418    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.470482 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.470429    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.471955 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.471943    2583 policy_none.go:50] "Start"
Oct 31 03:27:56.471994 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.471959    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 03:27:56.471994 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.471973    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 03:27:56.473903 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.473892    2583 policy_none.go:44] "Start"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.517958    2583 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.521718    2583 manager.go:341] "Starting Device Plugin manager"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.521755    2583 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.521767    2583 server.go:87] "Starting device plugin registration server"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.522082    2583 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.522096    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.522478    2583 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.522584    2583 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.522593    2583 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.523522    2583 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.523589    2583 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.535178    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.536375    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.536399    2583 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.536424    2583 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 03:27:56.536896 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.536465    2583 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 03:27:56.577470 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.577439    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-108.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 03:27:56.577470 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.577442    2583 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-108.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 03:27:56.622926 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.622807    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.623840 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.623820    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.623959 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.623855    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.623959 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.623866    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.623959 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.623895    2583 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.624105 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.622890    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-108.us-west-2.compute.internal.18e37f82d00df80f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-108.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-108.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 03:27:56.436641807 +0000 UTC m=+0.355803040,LastTimestamp:2026-10-31 03:27:56.436641807 +0000 UTC m=+0.355803040,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-108.us-west-2.compute.internal,}"
Oct 31 03:27:56.634704 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.634679    2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 31 03:27:56.634825 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.634726    2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-108.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 31 03:27:56.637333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.637310    2583 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"]
Oct 31 03:27:56.637432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.637380    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.638301 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.638279    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.638375 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.638314    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.638375 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.638326    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.640632 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.640618    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.640775 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.640761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.640813 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.640792    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.641531 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.641514    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.641603 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.641551    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.641603 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.641559    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.641603 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.641583    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.641603 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.641594    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.641603 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.641565    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.643869 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.643852    2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 03:27:56.643929 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.643907    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bdqfj"
Oct 31 03:27:56.643983 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.643969    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.644022 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.643998    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:56.644991 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.644971    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:56.645090 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.645003    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:56.645090 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.645017    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:56.647088 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.647066    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.670030 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.670007    2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.672565 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.672547    2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.729357 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.729326    2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:56.729868 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.729850    2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:56.735980 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.735960    2583 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.736085 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.735989    2583 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-108.us-west-2.compute.internal\": node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:56.748145 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.748119    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.748278 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.748154    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.748278 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.748180    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79d038c1345182a8a45cdaeaa396f9d8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"79d038c1345182a8a45cdaeaa396f9d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.748278 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.748220    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.849167 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.849137    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79d038c1345182a8a45cdaeaa396f9d8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"79d038c1345182a8a45cdaeaa396f9d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.849356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.849177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.849356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.849262    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.849356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.849269    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79d038c1345182a8a45cdaeaa396f9d8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"79d038c1345182a8a45cdaeaa396f9d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.862532 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.862511    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:56.963029 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:56.962951    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:56.973577 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.973542    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:56.976708 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:56.976691    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:57.063999 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.063813    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.164709 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.164664    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.265688 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.265598    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.366816 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.366783    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.378827 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.378798    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 03:27:57.379031 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.379006    2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 03:27:57.379106 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.379009    2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 03:27:57.379209 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.379033    2583 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aad67609fd0bc481f905aa31524c4d83-10f2963fa6f7f4fa.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-129-108.us-west-2.compute.internal.18e37f82d1fd3d64\": read tcp 10.0.129.108:37732->3.151.147.33:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-129-108.us-west-2.compute.internal.18e37f82d1fd3d64  default   5413 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-108.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-108.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-108.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 03:27:56 +0000 UTC,LastTimestamp:2026-10-31 03:27:56.638318844 +0000 UTC m=+0.557480077,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-108.us-west-2.compute.internal,}"
Oct 31 03:27:57.467674 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.467630    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.568230 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.568117    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.584800 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.584766    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:57.585734 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.585715    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:57.585838 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.585748    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:57.585838 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.585762    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:57.649440 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.649392    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 03:22:57 +0000 UTC" deadline="2028-07-27 04:59:00.060437588 +0000 UTC"
Oct 31 03:27:57.649440 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:57.649431    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15241h31m2.411012071s"
Oct 31 03:27:57.669029 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.668998    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.770256 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.770201    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.870425 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.870326    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:57.971061 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:57.971025    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:27:58.027222 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.027181    2583 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:58.046100 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.046073    2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:58.108559 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.108531    2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:27:58.110532 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.110514    2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:27:58.209506 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.209479    2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:27:58.209786 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.209771    2583 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:58.284753 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.284700    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d038c1345182a8a45cdaeaa396f9d8.slice/crio-ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c WatchSource:0}: Error finding container ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c: Status 404 returned error can't find the container with id ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c
Oct 31 03:27:58.287793 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.287762    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00518c13840ee186767099c11c28d53.slice/crio-27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59 WatchSource:0}: Error finding container 27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59: Status 404 returned error can't find the container with id 27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59
Oct 31 03:27:58.290369 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.290347    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:27:58.424273 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.424167    2583 apiserver.go:52] "Watching apiserver"
Oct 31 03:27:58.434669 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.434631    2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:58.445053 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.445012    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 03:27:58.528157 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.528121    2583 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 03:27:58.528506 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.528487    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx","openshift-dns/node-resolver-stklk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal","openshift-network-operator/iptables-alerter-zv4qg","kube-system/konnectivity-agent-p8r6z","openshift-multus/multus-zfvc8","openshift-ovn-kubernetes/ovnkube-node-rxfjs","kube-system/global-pull-secret-syncer-xgj98","openshift-cluster-node-tuning-operator/tuned-kggl4","openshift-image-registry/node-ca-6m89w","openshift-multus/multus-additional-cni-plugins-mvgh4","openshift-multus/network-metrics-daemon-cwdx2","openshift-network-diagnostics/network-check-target-7qbtw"]
Oct 31 03:27:58.532630 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.532610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.534813 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.534781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:58.534950 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.534862    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:27:58.537119 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.537093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.539506 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.539488    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:27:58.539581 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.539548    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:27:58.542166 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.541959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.545149 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.545132    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.549270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.549252    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.549405 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.549312    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:27:58.551687 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.551662    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.554575 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.554558    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.557248 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.557226    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.559634 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559612    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-modprobe-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.559739 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559647    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-conf\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.559739 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559663    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.559739 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559679    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-systemd\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.559739 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-var-lib-kubelet\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.559739 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-tmp\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560002 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvntl\" (UniqueName: \"kubernetes.io/projected/981be9db-6718-42fc-9076-8262bc218c66-kube-api-access-wvntl\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.560002 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559848    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c04cb90b-5446-4325-a368-a54295d49f33-konnectivity-ca\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.560002 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-run\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560002 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559914    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-269hl\" (UniqueName: \"kubernetes.io/projected/c80ccf99-2313-400b-8477-2f14a8d08c50-kube-api-access-269hl\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560002 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981be9db-6718-42fc-9076-8262bc218c66-tmp-dir\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.560002 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.559981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-dbus\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560037    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c04cb90b-5446-4325-a368-a54295d49f33-agent-certs\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-lib-modules\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysconfig\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560153    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-tuned\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560242    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560279    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9987cbad-f4ba-43be-91f7-f840a6fe0411-host-slash\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.560333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-kubernetes\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560336    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-sys\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560371    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpwr\" (UniqueName: \"kubernetes.io/projected/9987cbad-f4ba-43be-91f7-f840a6fe0411-kube-api-access-5kpwr\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560395    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-host\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560420    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-kubelet-config\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560444    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560479    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpm6t\" (UniqueName: \"kubernetes.io/projected/111ff747-5e37-43eb-8488-ea2ebe3432cc-kube-api-access-wpm6t\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560516    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9987cbad-f4ba-43be-91f7-f840a6fe0411-iptables-alerter-script\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.560715 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.560556    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/981be9db-6718-42fc-9076-8262bc218c66-hosts-file\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.562211 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.562192    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.564772 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.564735    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"f00518c13840ee186767099c11c28d53","Type":"ContainerStarted","Data":"27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59"}
Oct 31 03:27:58.564875 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.564788    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"79d038c1345182a8a45cdaeaa396f9d8","Type":"ContainerStarted","Data":"ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c"}
Oct 31 03:27:58.585802 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.585778    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.585802 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.585798    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zvcgd\""
Oct 31 03:27:58.585928 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.585813    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.589581 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.589566    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hztlx\""
Oct 31 03:27:58.589651 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.589620    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.589706 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.589648    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.594608 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.594585    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2rc66\""
Oct 31 03:27:58.594672 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.594592    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.594672 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.594592    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.594672 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.594651    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 03:27:58.597440 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.597424    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 03:27:58.597681 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.597668    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 03:27:58.597812 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.597799    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8nlcv\""
Oct 31 03:27:58.606654 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.604573    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.606654 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.604628    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 03:27:58.606654 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.604628    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-dlcmg\""
Oct 31 03:27:58.606654 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.604769    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.606654 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.604775    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 03:27:58.647501 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.647465    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 03:27:58.661512 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661482    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:58.661598 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-kubernetes\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.661598 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-sys\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.661598 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661567    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762debf9-cb16-4dc7-9923-c345e031f290-serviceca\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.661598 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661592    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.661758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-socket-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.661758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661656    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-kubernetes\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.661758 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.661664    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:58.661758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661691    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-sys\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.661758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661691    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpwr\" (UniqueName: \"kubernetes.io/projected/9987cbad-f4ba-43be-91f7-f840a6fe0411-kube-api-access-5kpwr\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.661758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661730    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-host\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.661791    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:27:59.161762495 +0000 UTC m=+3.080923745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661854    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cnibin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-k8s-cni-cncf-io\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661897    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-bin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661960    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-host\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.661962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-kubelet-config\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.662042 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662023    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-kubelet-config\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662063    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpm6t\" (UniqueName: \"kubernetes.io/projected/111ff747-5e37-43eb-8488-ea2ebe3432cc-kube-api-access-wpm6t\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.662229    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9987cbad-f4ba-43be-91f7-f840a6fe0411-iptables-alerter-script\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.662282    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:59.162265923 +0000 UTC m=+3.081427163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-kubelet\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662380    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-kubelet\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662397    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-netns\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.662426 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-modprobe-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662440    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-conf\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-var-lib-kubelet\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-tmp\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662537    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-modprobe-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662571    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-conf\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662582    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-var-lib-kubelet\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662599    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbdbd\" (UniqueName: \"kubernetes.io/projected/762debf9-cb16-4dc7-9923-c345e031f290-kube-api-access-wbdbd\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662629    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cni-binary-copy\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c04cb90b-5446-4325-a368-a54295d49f33-konnectivity-ca\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662676    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-run\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662764    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-run\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-269hl\" (UniqueName: \"kubernetes.io/projected/c80ccf99-2313-400b-8477-2f14a8d08c50-kube-api-access-269hl\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662779    2583 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981be9db-6718-42fc-9076-8262bc218c66-tmp-dir\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-system-cni-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-etc-kubernetes\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662882    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-var-lib-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.662892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-node-log\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-dbus\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgq7n\" (UniqueName: \"kubernetes.io/projected/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-kube-api-access-xgq7n\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662945    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9987cbad-f4ba-43be-91f7-f840a6fe0411-iptables-alerter-script\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.662957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-hostroot\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-systemd-units\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663093    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663096    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-dbus\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663115    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981be9db-6718-42fc-9076-8262bc218c66-tmp-dir\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c04cb90b-5446-4325-a368-a54295d49f33-agent-certs\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663154    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-lib-modules\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663232    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c04cb90b-5446-4325-a368-a54295d49f33-konnectivity-ca\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663312    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cnibin\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663399    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-system-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663416    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-sys-fs\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.663716 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663432    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663466    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-lib-modules\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663479    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-socket-dir-parent\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663564    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-etc-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663605    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x497z\" (UniqueName: \"kubernetes.io/projected/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-kube-api-access-x497z\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663670    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663719    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9987cbad-f4ba-43be-91f7-f840a6fe0411-host-slash\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-multus\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663772    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9987cbad-f4ba-43be-91f7-f840a6fe0411-host-slash\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663823    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-registration-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-script-lib\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663921    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/981be9db-6718-42fc-9076-8262bc218c66-hosts-file\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.664569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.663986    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-daemon-config\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-multus-certs\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664064    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/981be9db-6718-42fc-9076-8262bc218c66-hosts-file\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-systemd\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvntl\" (UniqueName: \"kubernetes.io/projected/981be9db-6718-42fc-9076-8262bc218c66-kube-api-access-wvntl\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-device-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664193    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-systemd\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-systemd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664260    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762debf9-cb16-4dc7-9923-c345e031f290-host\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664318    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-netns\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-slash\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664442    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-bin\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-netd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664532    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-env-overrides\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-ovn\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664581    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-config\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664622    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sthkd\" (UniqueName: \"kubernetes.io/projected/2467cba4-f180-4a23-bb6a-5a67c8e86df4-kube-api-access-sthkd\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664694    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gnm2\" (UniqueName: \"kubernetes.io/projected/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kube-api-access-7gnm2\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-log-socket\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664738    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysconfig\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664788    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysconfig\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664814    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-tuned\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.665691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.664960    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-conf-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.666635 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.666616    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-tmp\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.666714 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.666669    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-tuned\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.666844 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.666825    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c04cb90b-5446-4325-a368-a54295d49f33-agent-certs\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.670540 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.670524    2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:58.689037 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.688984    2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 03:27:58.689124 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.689045    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-96ws4"
Oct 31 03:27:58.694431 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694414    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 03:27:58.694510 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694458    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 03:27:58.694510 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694485    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 03:27:58.694614 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694510    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.694614 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694531    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.694614 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694568    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 03:27:58.694778 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694757    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.694778 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694760    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 03:27:58.694778 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694776    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hp8t5\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694808    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694816    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694842    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7b9lc\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694852    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694853    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694908    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hp5ds\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694918    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 03:27:58.694999 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.694932    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9sxx\""
Oct 31 03:27:58.695309 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.695081    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 03:27:58.722481 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.722456    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:58.722481 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.722482    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:58.722671 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.722495    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:58.722671 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:58.722567    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:27:59.222547586 +0000 UTC m=+3.141708829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:58.723849 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.723822    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpwr\" (UniqueName: \"kubernetes.io/projected/9987cbad-f4ba-43be-91f7-f840a6fe0411-kube-api-access-5kpwr\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.723951 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.723862    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpm6t\" (UniqueName: \"kubernetes.io/projected/111ff747-5e37-43eb-8488-ea2ebe3432cc-kube-api-access-wpm6t\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:58.724159 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.724138    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-269hl\" (UniqueName: \"kubernetes.io/projected/c80ccf99-2313-400b-8477-2f14a8d08c50-kube-api-access-269hl\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.725311 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.725271    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvntl\" (UniqueName: \"kubernetes.io/projected/981be9db-6718-42fc-9076-8262bc218c66-kube-api-access-wvntl\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.765786 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.765752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-conf-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.765934 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.765810    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762debf9-cb16-4dc7-9923-c345e031f290-serviceca\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.765934 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.765883    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-conf-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766030 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.765941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766030 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.765994    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-socket-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.766133 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766027    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cnibin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766133 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-k8s-cni-cncf-io\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766133 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766082    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766133 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766078    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-bin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766137    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cnibin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766166    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-socket-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766176    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-k8s-cni-cncf-io\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766142    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-bin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-kubelet\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766178    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-kubelet\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766262    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-kubelet\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766325 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766314    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-kubelet\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-netns\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766381    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbdbd\" (UniqueName: \"kubernetes.io/projected/762debf9-cb16-4dc7-9923-c345e031f290-kube-api-access-wbdbd\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cni-binary-copy\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-system-cni-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766450    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-netns\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-etc-kubernetes\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766483    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762debf9-cb16-4dc7-9923-c345e031f290-serviceca\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766494    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-system-cni-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766485    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-var-lib-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766518    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-etc-kubernetes\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-node-log\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766556    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-var-lib-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgq7n\" (UniqueName: \"kubernetes.io/projected/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-kube-api-access-xgq7n\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766583    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-node-log\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-hostroot\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-systemd-units\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766625    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.766794 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766652    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-hostroot\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cnibin\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766667    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-systemd-units\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766700    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766703    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-system-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766721    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cnibin\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766741    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-sys-fs\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766750    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-system-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766770    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766790    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-sys-fs\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766798    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-socket-dir-parent\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766849    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-etc-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766828    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766885    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-etc-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766872    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-socket-dir-parent\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.767660 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x497z\" (UniqueName: \"kubernetes.io/projected/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-kube-api-access-x497z\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-multus\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766978    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-registration-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.766993    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cni-binary-copy\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-script-lib\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767055    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767061    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-registration-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-daemon-config\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767154    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-multus-certs\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767209    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-multus-certs\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-device-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767285    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-systemd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767312    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762debf9-cb16-4dc7-9923-c345e031f290-host\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.768380 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-netns\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767341    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-slash\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-bin\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767401    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767417    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-netd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767435    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-env-overrides\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767453    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-device-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767474    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-ovn\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767496    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-netns\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767502    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-config\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767496    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-systemd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767548    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762debf9-cb16-4dc7-9923-c345e031f290-host\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767562    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthkd\" (UniqueName: \"kubernetes.io/projected/2467cba4-f180-4a23-bb6a-5a67c8e86df4-kube-api-access-sthkd\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767577    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-bin\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.768976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gnm2\" (UniqueName: \"kubernetes.io/projected/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kube-api-access-7gnm2\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-log-socket\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767629    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-ovn\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767685    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-slash\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767534    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-netd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767732    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767741    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-script-lib\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767840    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-daemon-config\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767894    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-log-socket\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.767939    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-multus\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.768053    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.768138    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-config\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.768841    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-env-overrides\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.769498 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.769454    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.828436 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.828397    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgq7n\" (UniqueName: \"kubernetes.io/projected/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-kube-api-access-xgq7n\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.828625 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.828586    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x497z\" (UniqueName: \"kubernetes.io/projected/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-kube-api-access-x497z\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.828625 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.828616    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbdbd\" (UniqueName: \"kubernetes.io/projected/762debf9-cb16-4dc7-9923-c345e031f290-kube-api-access-wbdbd\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.828982 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.828961    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gnm2\" (UniqueName: \"kubernetes.io/projected/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kube-api-access-7gnm2\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.829146 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.829128    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthkd\" (UniqueName: \"kubernetes.io/projected/2467cba4-f180-4a23-bb6a-5a67c8e86df4-kube-api-access-sthkd\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.844335 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.844320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:27:58.853898 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.853877    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-stklk"
Oct 31 03:27:58.855084 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.855058    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80ccf99_2313_400b_8477_2f14a8d08c50.slice/crio-3d43c00c7bec8ebf0fde394b3e01fb50effc56a74630c483ba9e9272d9ebe096 WatchSource:0}: Error finding container 3d43c00c7bec8ebf0fde394b3e01fb50effc56a74630c483ba9e9272d9ebe096: Status 404 returned error can't find the container with id 3d43c00c7bec8ebf0fde394b3e01fb50effc56a74630c483ba9e9272d9ebe096
Oct 31 03:27:58.861539 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.861521    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:27:58.862692 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.862671    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod981be9db_6718_42fc_9076_8262bc218c66.slice/crio-3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7 WatchSource:0}: Error finding container 3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7: Status 404 returned error can't find the container with id 3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7
Oct 31 03:27:58.866156 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.866137    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:27:58.869969 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.869947    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9987cbad_f4ba_43be_91f7_f840a6fe0411.slice/crio-2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423 WatchSource:0}: Error finding container 2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423: Status 404 returned error can't find the container with id 2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423
Oct 31 03:27:58.872230 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.872199    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zfvc8"
Oct 31 03:27:58.875006 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.874985    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04cb90b_5446_4325_a368_a54295d49f33.slice/crio-34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62 WatchSource:0}: Error finding container 34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62: Status 404 returned error can't find the container with id 34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62
Oct 31 03:27:58.881249 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.881227    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2467cba4_f180_4a23_bb6a_5a67c8e86df4.slice/crio-5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850 WatchSource:0}: Error finding container 5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850: Status 404 returned error can't find the container with id 5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850
Oct 31 03:27:58.882021 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.882005    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:27:58.890752 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.890731    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a7862a2_9e10_46d0_b7e9_929eb6158d3d.slice/crio-98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739 WatchSource:0}: Error finding container 98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739: Status 404 returned error can't find the container with id 98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739
Oct 31 03:27:58.896013 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.895996    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:27:58.904901 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.904879    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762debf9_cb16_4dc7_9923_c345e031f290.slice/crio-5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18 WatchSource:0}: Error finding container 5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18: Status 404 returned error can't find the container with id 5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18
Oct 31 03:27:58.913148 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.913128    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:27:58.920301 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:58.920281    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:27:58.922168 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.922147    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87c006a7_67b8_4a1d_af88_d6a66fe6e2b5.slice/crio-0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5 WatchSource:0}: Error finding container 0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5: Status 404 returned error can't find the container with id 0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5
Oct 31 03:27:58.930459 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:27:58.930436    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd31cb71_d1cb_4377_b6ef_2c9e08b1c887.slice/crio-cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6 WatchSource:0}: Error finding container cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6: Status 404 returned error can't find the container with id cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6
Oct 31 03:27:59.170790 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.170696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:27:59.170790 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.170742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:27:59.170978 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.170853    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:59.170978 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.170854    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:59.170978 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.170908    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:00.1708955 +0000 UTC m=+4.090056721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:59.170978 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.170922    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:28:00.17091575 +0000 UTC m=+4.090076972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:59.272422 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.272371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:27:59.272618 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.272598    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:59.272675 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.272624    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:59.272675 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.272636    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:59.272749 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:27:59.272700    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:28:00.272681733 +0000 UTC m=+4.191842978 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:59.548169 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.548075    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zv4qg" event={"ID":"9987cbad-f4ba-43be-91f7-f840a6fe0411","Type":"ContainerStarted","Data":"2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423"}
Oct 31 03:27:59.549696 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.549645    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerStarted","Data":"cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6"}
Oct 31 03:27:59.551910 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.551849    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5"}
Oct 31 03:27:59.553262 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.553209    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6m89w" event={"ID":"762debf9-cb16-4dc7-9923-c345e031f290","Type":"ContainerStarted","Data":"5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18"}
Oct 31 03:27:59.555020 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.554981    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p8r6z" event={"ID":"c04cb90b-5446-4325-a368-a54295d49f33","Type":"ContainerStarted","Data":"34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62"}
Oct 31 03:27:59.556623 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.556601    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-stklk" event={"ID":"981be9db-6718-42fc-9076-8262bc218c66","Type":"ContainerStarted","Data":"3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7"}
Oct 31 03:27:59.558100 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.558073    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kggl4" event={"ID":"c80ccf99-2313-400b-8477-2f14a8d08c50","Type":"ContainerStarted","Data":"3d43c00c7bec8ebf0fde394b3e01fb50effc56a74630c483ba9e9272d9ebe096"}
Oct 31 03:27:59.559680 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.559648    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx" event={"ID":"6a7862a2-9e10-46d0-b7e9-929eb6158d3d","Type":"ContainerStarted","Data":"98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739"}
Oct 31 03:27:59.561143 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.561089    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zfvc8" event={"ID":"2467cba4-f180-4a23-bb6a-5a67c8e86df4","Type":"ContainerStarted","Data":"5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850"}
Oct 31 03:27:59.693686 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.693601    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 03:22:59 +0000 UTC" deadline="2028-07-15 04:36:21.580138016 +0000 UTC"
Oct 31 03:27:59.693686 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:27:59.693670    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14953h8m21.886472768s"
Oct 31 03:28:00.180774 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.180375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:00.180774 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.180434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:00.180774 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.180571    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:00.180774 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.180634    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:02.18061513 +0000 UTC m=+6.099776354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:00.180774 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.180706    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:00.180774 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.180742    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:28:02.180729371 +0000 UTC m=+6.099890597 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:00.281564 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.281015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:00.281564 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.281377    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:28:00.281564 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.281417    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:28:00.281564 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.281453    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:00.281564 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.281522    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:28:02.281501093 +0000 UTC m=+6.200662321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:00.542600 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.542520    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:00.542774 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.542667    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:00.542774 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.542761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:00.542910 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.542854    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:00.542973 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.542938    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:00.543056 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:00.543025    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:00.693906 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.693835    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 03:22:59 +0000 UTC" deadline="2028-04-04 14:09:02.796345173 +0000 UTC"
Oct 31 03:28:00.693906 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:00.693904    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12514h41m2.10244668s"
Oct 31 03:28:02.200195 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:02.199815    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:02.200195 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:02.199873    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:02.200195 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.199998    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:02.200195 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.200059    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:06.200041251 +0000 UTC m=+10.119202495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:02.200195 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.200129    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:02.200195 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.200165    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:28:06.200152478 +0000 UTC m=+10.119313719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:02.301411 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:02.301097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:02.301411 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.301258    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:28:02.301411 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.301274    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:28:02.301411 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.301284    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:02.301411 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.301335    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:28:06.301319514 +0000 UTC m=+10.220480755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:02.540668 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:02.540146    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:02.540668 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.540308    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:02.540668 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:02.540386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:02.540668 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.540467    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:02.540668 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:02.540533    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:02.540668 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:02.540618    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:03.595672 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:03.595248    2583 generic.go:356] "Generic (PLEG): container finished" podID="f00518c13840ee186767099c11c28d53" containerID="2edee0761bcab0ae5348f9e21563021f4888a7d6273ec165031993fa8f22fd0b" exitCode=0
Oct 31 03:28:03.595672 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:03.595500    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"f00518c13840ee186767099c11c28d53","Type":"ContainerDied","Data":"2edee0761bcab0ae5348f9e21563021f4888a7d6273ec165031993fa8f22fd0b"}
Oct 31 03:28:03.611081 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:03.610525    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"79d038c1345182a8a45cdaeaa396f9d8","Type":"ContainerStarted","Data":"71e163ea2d8bd26853bae5559fcb77428423dcf3444b5e99232cb4abf391a558"}
Oct 31 03:28:03.616298 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:03.615321    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="bea426325ad284005770e14c14fcf4f720ef64809f7458fa3e0bf51d8f1ace70" exitCode=0
Oct 31 03:28:03.616298 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:03.615377    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerDied","Data":"bea426325ad284005770e14c14fcf4f720ef64809f7458fa3e0bf51d8f1ace70"}
Oct 31 03:28:03.953857 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:03.953737    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal" podStartSLOduration=5.953716391 podStartE2EDuration="5.953716391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:03.830359451 +0000 UTC m=+7.749520698" watchObservedRunningTime="2026-10-31 03:28:03.953716391 +0000 UTC m=+7.872877629"
Oct 31 03:28:04.540024 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:04.539392    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:04.540024 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:04.539400    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:04.540024 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:04.539555    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:04.540024 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:04.539576    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:04.540024 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:04.539714    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:04.540024 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:04.539900    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:04.634587 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:04.634544    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"f00518c13840ee186767099c11c28d53","Type":"ContainerStarted","Data":"e38706dabbe0e2e6f8d08fcfc28ccd783837cff073c760b3e44a5b7fbe865010"}
Oct 31 03:28:04.640530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:04.640499    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p8r6z" event={"ID":"c04cb90b-5446-4325-a368-a54295d49f33","Type":"ContainerStarted","Data":"ba2d8484ee2f62114dafbf333368a153eb26e003c26521a0be20c325cc068774"}
Oct 31 03:28:04.870203 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:04.870060    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" podStartSLOduration=6.870036929 podStartE2EDuration="6.870036929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:04.753063538 +0000 UTC m=+8.672224802" watchObservedRunningTime="2026-10-31 03:28:04.870036929 +0000 UTC m=+8.789198176"
Oct 31 03:28:05.648461 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:05.648416    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6m89w" event={"ID":"762debf9-cb16-4dc7-9923-c345e031f290","Type":"ContainerStarted","Data":"bff92968a11ef9176a13c28180f6bd7677de6d6812963d55a6cbfcc072c116ef"}
Oct 31 03:28:05.763474 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:05.763332    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p8r6z" podStartSLOduration=3.61498608 podStartE2EDuration="8.763308817s" totalImagesPullingTime="5.148322737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:04.870620694 +0000 UTC m=+8.789781931" watchObservedRunningTime="2026-10-31 03:28:05.763308817 +0000 UTC m=+9.682470064"
Oct 31 03:28:06.246602 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.246561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:06.246787 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.246622    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:06.246787 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.246768    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:06.246924 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.246836    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.246816155 +0000 UTC m=+18.165977398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:06.247303 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.247258    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:06.247426 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.247340    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.247321196 +0000 UTC m=+18.166482438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:06.348336 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.348296    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:06.348528 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.348497    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:28:06.348528 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.348518    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:28:06.348644 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.348532    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:06.348644 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.348606    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.348586589 +0000 UTC m=+18.267747843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:06.540582 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.540486    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:06.540781 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.540627    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:06.540781 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.540700    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:06.540781 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.540773    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:06.540969 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.540844    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:06.540969 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:06.540924    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:06.653594 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:06.653556    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx" event={"ID":"6a7862a2-9e10-46d0-b7e9-929eb6158d3d","Type":"ContainerStarted","Data":"398171a444777f1bc91b88b4c6d792245e519a57be00e0510b709fd541b6dd70"}
Oct 31 03:28:07.236255 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:07.236202    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:28:08.539779 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.539739    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:08.540334 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:08.539890    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:08.540334 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.539967    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:08.540334 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:08.540041    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:08.540334 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.540106    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:08.540334 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:08.540188    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:08.651010 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.650978    2583 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 03:28:08.661815 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.661769    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-stklk" event={"ID":"981be9db-6718-42fc-9076-8262bc218c66","Type":"ContainerStarted","Data":"83bf6bcc3cf075b95c51df72260ba9007fe7565638b2c1337051d6a651d05cc3"}
Oct 31 03:28:08.665686 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.665658    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx" event={"ID":"6a7862a2-9e10-46d0-b7e9-929eb6158d3d","Type":"ContainerStarted","Data":"36659c481cadd08444c816f422073434ab7fd4acc00242bffdc341a0c9e450dd"}
Oct 31 03:28:08.667559 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.667527    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zfvc8" event={"ID":"2467cba4-f180-4a23-bb6a-5a67c8e86df4","Type":"ContainerStarted","Data":"76e317a7d16c19153f1e3bcbf0f839ce4083d0706d15fc27a51df8ca60064cbb"}
Oct 31 03:28:08.777973 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.777923    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6m89w" podStartSLOduration=5.38481194 podStartE2EDuration="11.777909689s" totalImagesPullingTime="6.393097749s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:05.764548695 +0000 UTC m=+9.683709923" watchObservedRunningTime="2026-10-31 03:28:08.777909689 +0000 UTC m=+12.697070934"
Oct 31 03:28:08.778235 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.778216    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-stklk" podStartSLOduration=2.215004129 podStartE2EDuration="11.778210498s" totalImagesPullingTime="9.563206369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:08.777581149 +0000 UTC m=+12.696742394" watchObservedRunningTime="2026-10-31 03:28:08.778210498 +0000 UTC m=+12.697371744"
Oct 31 03:28:08.897811 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.897519    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-zfvc8" podStartSLOduration=3.127142636 podStartE2EDuration="11.897501476s" totalImagesPullingTime="8.77035884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:08.89666725 +0000 UTC m=+12.815828496" watchObservedRunningTime="2026-10-31 03:28:08.897501476 +0000 UTC m=+12.816662721"
Oct 31 03:28:08.931192 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.931156    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:28:08.931885 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:08.931859    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:28:09.534655 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:09.534544    2583 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T03:28:08.651001429Z","UUID":"d0de1a92-06aa-435e-bfff-b12413d33d3e","Handler":null,"Name":"","Endpoint":""}
Oct 31 03:28:09.536638 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:09.536238    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 03:28:09.536638 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:09.536272    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 03:28:09.671713 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:09.671675    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="e0638635444f0a376a20affeb6efb9d0e8f4c3140017f33c2710ee67482dcb8a" exitCode=0
Oct 31 03:28:09.672143 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:09.671944    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerDied","Data":"e0638635444f0a376a20affeb6efb9d0e8f4c3140017f33c2710ee67482dcb8a"}
Oct 31 03:28:09.673457 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:09.673191    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:28:10.536833 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:10.536747    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:10.536833 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:10.536812    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:10.536833 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:10.536827    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:10.537081 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:10.536927    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:10.537081 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:10.537026    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:10.537169 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:10.537114    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:10.676246 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:10.676188    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zv4qg" event={"ID":"9987cbad-f4ba-43be-91f7-f840a6fe0411","Type":"ContainerStarted","Data":"12b29ae293642f4f1b6a4de6872a58a3424598cc4659f9bb07798b2003b75a01"}
Oct 31 03:28:10.783806 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:10.783743    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zv4qg" podStartSLOduration=3.889358309 podStartE2EDuration="13.783722979s" totalImagesPullingTime="9.89436467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:10.783458526 +0000 UTC m=+14.702619769" watchObservedRunningTime="2026-10-31 03:28:10.783722979 +0000 UTC m=+14.702884224"
Oct 31 03:28:11.681305 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:11.681197    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="e62d89391360705331a9940f4fe59d7c6bc3c51b0605b70c1ae2c07dbc72fc17" exitCode=0
Oct 31 03:28:11.681991 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:11.681309    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerDied","Data":"e62d89391360705331a9940f4fe59d7c6bc3c51b0605b70c1ae2c07dbc72fc17"}
Oct 31 03:28:11.683556 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:11.683528    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kggl4" event={"ID":"c80ccf99-2313-400b-8477-2f14a8d08c50","Type":"ContainerStarted","Data":"d2cb2d8ec6152288c5c738442047a19ddd0231eec1ac7109d58150a150339ad3"}
Oct 31 03:28:11.685987 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:11.685958    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx" event={"ID":"6a7862a2-9e10-46d0-b7e9-929eb6158d3d","Type":"ContainerStarted","Data":"2302097c3f8348f3c1737a874b05d9527290e3b7d6796c2c16638b85c197a37c"}
Oct 31 03:28:11.906408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:11.906349    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kggl4" podStartSLOduration=2.943743598 podStartE2EDuration="14.90633198s" totalImagesPullingTime="11.962588382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:11.905951517 +0000 UTC m=+15.825112760" watchObservedRunningTime="2026-10-31 03:28:11.90633198 +0000 UTC m=+15.825493222"
Oct 31 03:28:12.016954 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:12.016833    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx" podStartSLOduration=3.492955466 podStartE2EDuration="15.016819418s" totalImagesPullingTime="11.523863952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:12.016375543 +0000 UTC m=+15.935536785" watchObservedRunningTime="2026-10-31 03:28:12.016819418 +0000 UTC m=+15.935980660"
Oct 31 03:28:12.507271 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:12.507173    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:12.537333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:12.537291    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:12.537333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:12.537320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:12.537552 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:12.537443    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:12.537552 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:12.537482    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:12.537668 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:12.537594    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:12.537717 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:12.537674    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:12.697500 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:12.697461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:13.694076 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:13.693869    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="577dc403f1a521e46c7e1b0168fc287bc8696a20d00c451c2ada03b7727198f0" exitCode=0
Oct 31 03:28:13.694292 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:13.693913    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerDied","Data":"577dc403f1a521e46c7e1b0168fc287bc8696a20d00c451c2ada03b7727198f0"}
Oct 31 03:28:14.318424 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:14.318385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:14.318424 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:14.318431    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:14.319051 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.318556    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:14.319051 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.318589    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:14.319051 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.318640    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:30.318619491 +0000 UTC m=+34.237780726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:14.319051 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.318657    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:28:30.318648823 +0000 UTC m=+34.237810043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:14.419519 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:14.419480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:14.419682 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.419645    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:28:14.419682 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.419667    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:28:14.419682 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.419677    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:14.419780 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.419727    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:28:30.419715174 +0000 UTC m=+34.338876395 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:14.540432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:14.540404    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:14.540432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:14.540433    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:14.540432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:14.540442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:14.540682 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.540509    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:14.540682 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.540582    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:14.540682 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:14.540630    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:15.271747 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.271710    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:15.412753 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.412528    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:15.701998 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.701916    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"1c7f51e80727c5418c7f429ee69efe7ceca69ec97f965dbfd9850f05b3a21138"}
Oct 31 03:28:15.702145 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.702034    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"08aad623deeda6b425b0aeb5821b91423de0db8ce4f440c424dec84aca4ab0d9"}
Oct 31 03:28:15.702145 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.702069    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"7a74c6bc2cb4887c23251fb842c3135102852171f7491c647bd3e1974c60d2b3"}
Oct 31 03:28:15.702145 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.702097    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"86b45166e1aa96c2ff98083cd59b49cf72b89489008018bddc8a32bf637e7588"}
Oct 31 03:28:15.702145 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.702125    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"a61936c8aaf1e56dcc99705ca7cba97808c06ee22820acfc5133fb86834ce767"}
Oct 31 03:28:15.702320 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:15.702150    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"f5e98d4bb3a18e288a9ca48a9d2aee3c6219e4f8b307a3b6c2c57b3124d496e5"}
Oct 31 03:28:16.539315 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:16.539282    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:16.539315 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:16.539299    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:16.539315 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:16.539284    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:16.539875 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:16.539387    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:16.539875 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:16.539534    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:16.539875 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:16.539624    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:16.706331 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:16.706298    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="9ad34fb6e06518693f7d2e5d963cce24b36e235bef371aae4556647d3e381aa1" exitCode=0
Oct 31 03:28:16.706483 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:16.706343    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerDied","Data":"9ad34fb6e06518693f7d2e5d963cce24b36e235bef371aae4556647d3e381aa1"}
Oct 31 03:28:17.293646 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:17.293615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:17.433732 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:17.433685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:17.711702 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:17.711600    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"2df789eb561f89304a5ed46895ae9f7ab6e882a3841993d456299be9a8f40aed"}
Oct 31 03:28:17.714370 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:17.714341    2583 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="010030d5385bf68e31959e271bde80917b00291b7ca6c1bf70b3f447507e402d" exitCode=0
Oct 31 03:28:17.714370 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:17.714366    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerDied","Data":"010030d5385bf68e31959e271bde80917b00291b7ca6c1bf70b3f447507e402d"}
Oct 31 03:28:18.539935 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:18.539899    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:18.540123 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:18.540013    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:18.540123 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:18.540024    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:18.540123 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:18.540040    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:18.540123 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:18.540114    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:18.540347 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:18.540173    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:18.719529 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:18.719491    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" event={"ID":"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887","Type":"ContainerStarted","Data":"64c2cdf885365b5059730f9ceae8be8d9d5de0b7bca5baf2ea82c3e418488763"}
Oct 31 03:28:18.880025 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:18.879920    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mvgh4" podStartSLOduration=7.555799303 podStartE2EDuration="21.879906747s" totalImagesPullingTime="14.324107444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:18.879310651 +0000 UTC m=+22.798471892" watchObservedRunningTime="2026-10-31 03:28:18.879906747 +0000 UTC m=+22.799067988"
Oct 31 03:28:19.106974 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:19.106932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:19.246088 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:19.246014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:19.725976 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:19.725727    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" event={"ID":"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5","Type":"ContainerStarted","Data":"866d9474a90bab8260f02d5e0d68e695a66faee13ee810e67116a9ec8f3ffe49"}
Oct 31 03:28:19.984175 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:19.984045    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs" podStartSLOduration=7.043060279 podStartE2EDuration="22.984032481s" totalImagesPullingTime="15.940972202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:19.937791112 +0000 UTC m=+23.856952355" watchObservedRunningTime="2026-10-31 03:28:19.984032481 +0000 UTC m=+23.903193723"
Oct 31 03:28:20.536983 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.536945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:20.537148 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.536999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:20.537148 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.536951    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:20.537148 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:20.537090    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:20.537295 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:20.537170    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:20.537295 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:20.537251    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:20.660617 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.660576    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:20.728698 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.728662    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:28:20.729122 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.728744    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:28:20.729122 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.728794    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:28:20.744487 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.744460    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:28:20.747675 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.747656    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:28:20.824507 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:20.824436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:21.499221 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:21.499172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:21.640728 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:21.640694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:21.790939 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:21.790853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:21.932129 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:21.932092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:22.073344 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.073253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:22.110918 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.110883    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7qbtw"]
Oct 31 03:28:22.111090 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.111031    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:22.111165 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:22.111113    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:22.113146 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.113035    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xgj98"]
Oct 31 03:28:22.113415 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.113392    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:22.113513 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:22.113495    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:22.113921 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.113900    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cwdx2"]
Oct 31 03:28:22.114007 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.114001    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:22.114101 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:22.114080    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:22.919187 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:22.919152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:23.059735 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:23.059704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:23.537382 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:23.537350    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:23.537382 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:23.537382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:23.537640 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:23.537440    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:23.537640 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:23.537454    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:23.537640 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:23.537547    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:23.537640 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:23.537627    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:23.560532 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:23.560495    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:25.106544 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:25.106516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:25.537249 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:25.537185    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:25.537249 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:25.537237    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:25.537448 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:25.537324    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:25.537448 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:25.537393    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:25.537512 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:25.537483    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:25.537559 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:25.537543    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:26.498242 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:26.498192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:26.649049 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:26.649019    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:26.790892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:26.790815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:27.537500 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:27.537469    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:27.537906 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:27.537471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:27.537906 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:27.537578    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:27.537906 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:27.537676    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:27.537906 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:27.537471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:27.537906 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:27.537757    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:28.854764 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:28.854571    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:28.993501 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:28.993470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:29.130149 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.130064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:29.444505 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.444434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:29.537257 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.537192    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:29.537257 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.537241    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:29.537465 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:29.537328    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:29.537465 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.537382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:29.537465 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:29.537452    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:29.537582 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:29.537512    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:29.590363 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.590329    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:29.748565 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.748488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:29.886242 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:29.886193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:30.025078 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:30.024989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:30.164037 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:30.164006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:30.305707 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:30.305633    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:30.338625 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:30.338588    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:30.338625 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:30.338629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:30.338824 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.338741    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:30.338824 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.338743    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:30.338824 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.338790    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret podName:24753e34-7be9-4a7e-8629-622b22fde606 nodeName:}" failed. No retries permitted until 2026-10-31 03:29:02.338778784 +0000 UTC m=+66.257940016 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret") pod "global-pull-secret-syncer-xgj98" (UID: "24753e34-7be9-4a7e-8629-622b22fde606") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:28:30.338824 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.338803    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs podName:111ff747-5e37-43eb-8488-ea2ebe3432cc nodeName:}" failed. No retries permitted until 2026-10-31 03:29:02.338797286 +0000 UTC m=+66.257958506 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs") pod "network-metrics-daemon-cwdx2" (UID: "111ff747-5e37-43eb-8488-ea2ebe3432cc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:28:30.439415 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:30.439376    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:30.439584 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.439515    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:28:30.439584 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.439532    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:28:30.439584 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.439543    2583 projected.go:196] Error preparing data for projected volume kube-api-access-98nsz for pod openshift-network-diagnostics/network-check-target-7qbtw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:30.439675 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:30.439589    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz podName:66b0fafb-8b2b-4b04-84ed-99e96902b9fd nodeName:}" failed. No retries permitted until 2026-10-31 03:29:02.439577004 +0000 UTC m=+66.358738237 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-98nsz" (UniqueName: "kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz") pod "network-check-target-7qbtw" (UID: "66b0fafb-8b2b-4b04-84ed-99e96902b9fd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:28:31.379284 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.379250    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:31.537038 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.537007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:31.537038 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.537041    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:31.537284 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.537044    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:31.537284 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:31.537135    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:31.537284 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:31.537199    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:31.537284 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:31.537281    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:31.583159 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.583125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:31.781845 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.781812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:31.965055 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:31.964977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:32.103196 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:32.103164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:32.242420 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:32.242338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:33.133669 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.133637    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-stklk_981be9db-6718-42fc-9076-8262bc218c66/dns-node-resolver/0.log"
Oct 31 03:28:33.380859 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.380827    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:33.537706 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.537614    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:33.537706 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.537650    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:33.537706 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.537654    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:33.537965 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:33.537731    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:33.537965 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:33.537800    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:33.537965 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:33.537860    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:33.584867 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.584832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:33.732753 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.732725    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6m89w_762debf9-cb16-4dc7-9923-c345e031f290/node-ca/0.log"
Oct 31 03:28:33.780748 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.780717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:33.981155 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:33.981072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:34.181095 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:34.181069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:35.185583 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.185535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:35.537029 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.536949    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:35.537180 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.536950    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:35.537180 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:35.537068    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:35.537180 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.536950    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:35.537180 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:35.537136    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:35.537349 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:35.537199    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:35.581628 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.581598    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:35.781910 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.781879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:35.980891 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:35.980808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:36.386049 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:36.385975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:37.189768 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.189732    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:37.331465 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.331426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:37.472433 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.472354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:37.536870 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.536829    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:37.536870 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.536877    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:37.537079 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.536845    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:37.537079 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:37.536959    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:37.537079 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:37.537031    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:37.537182 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:37.537102    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:37.608767 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.608734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:37.746049 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.745971    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:37.881676 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:37.881639    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:38.024300 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:38.024219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:38.266741 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:38.266712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:38.410015 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:38.409936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:38.555774 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:38.555735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:38.691756 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:38.691682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:38.831584 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:38.831547    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:39.188992 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.188910    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:39.537125 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.537041    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:39.537371 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.537042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:39.537371 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:39.537151    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:39.537371 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:39.537256    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:39.537371 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.537041    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:39.537534 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:39.537371    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:39.580220 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.580186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:39.785828 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.785793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:39.982972 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:39.982884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:40.380964 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:40.380929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:40.580387 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:40.580354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:41.537527 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:41.537486    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:41.537527 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:41.537531    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:41.537749 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:41.537500    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:41.537749 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:41.537622    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:41.537749 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:41.537674    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:41.537842 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:41.537770    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:41.651758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:41.651729    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:41.787762 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:41.787682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:43.536942 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:43.536906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:43.537486 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:43.536906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:43.537486 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:43.537037    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xgj98" podUID="24753e34-7be9-4a7e-8629-622b22fde606"
Oct 31 03:28:43.537486 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:43.536906    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:43.537486 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:43.537086    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7qbtw" podUID="66b0fafb-8b2b-4b04-84ed-99e96902b9fd"
Oct 31 03:28:43.537486 ip-10-0-129-108 kubenswrapper[2583]: E1031 03:28:43.537162    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwdx2" podUID="111ff747-5e37-43eb-8488-ea2ebe3432cc"
Oct 31 03:28:43.937038 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:43.936960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:44.080599 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.080568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:44.324625 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.324591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:44.461236 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.461186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:44.834007 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.833979    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeReady"
Oct 31 03:28:44.834412 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.834128    2583 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 03:28:44.942534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.942504    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8sd9m"]
Oct 31 03:28:44.944998 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.944973    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:44.988761 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.988724    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gzcxv"]
Oct 31 03:28:44.990815 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.990799    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:44.994424 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.994402    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 03:28:44.994710 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.994693    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fs258\""
Oct 31 03:28:44.994797 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:44.994776    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 03:28:45.004627 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.004591    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8sd9m"]
Oct 31 03:28:45.005173 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.005153    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gzcxv"]
Oct 31 03:28:45.031691 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.031608    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-94ztd"]
Oct 31 03:28:45.033565 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.033547    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.042376 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.042354    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-config-volume\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.042484 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.042409    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-metrics-tls\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.042528 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.042492    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95pf\" (UniqueName: \"kubernetes.io/projected/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-kube-api-access-l95pf\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.042528 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.042525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-tmp-dir\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.090127 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.090031    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-94ztd"]
Oct 31 03:28:45.095809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095759    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 03:28:45.095809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095789    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 03:28:45.096031 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095842    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 03:28:45.096031 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095863    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 03:28:45.096031 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095955    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 03:28:45.096031 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095965    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 03:28:45.096031 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.095991    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 03:28:45.096222 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.096064    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-dvnpq\""
Oct 31 03:28:45.096222 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.096117    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdz67\""
Oct 31 03:28:45.143093 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143053    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f88ad0-2936-4d56-9fef-a74e38d0c770-data-volume\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.143093 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-cert\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.143300 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-config-volume\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.143300 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143252    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-metrics-tls\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.143300 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42f88ad0-2936-4d56-9fef-a74e38d0c770-crio-socket\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.143401 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42f88ad0-2936-4d56-9fef-a74e38d0c770-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.143401 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143329    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6hs9\" (UniqueName: \"kubernetes.io/projected/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-api-access-q6hs9\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.143488 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95pf\" (UniqueName: \"kubernetes.io/projected/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-kube-api-access-l95pf\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.143525 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.143525 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143518    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77fcb\" (UniqueName: \"kubernetes.io/projected/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-kube-api-access-77fcb\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.143604 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-tmp-dir\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.143792 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143775    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-config-volume\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.143866 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.143805    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-tmp-dir\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.147074 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.147056    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-metrics-tls\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.200707 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.200670    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95pf\" (UniqueName: \"kubernetes.io/projected/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-kube-api-access-l95pf\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.244197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f88ad0-2936-4d56-9fef-a74e38d0c770-data-volume\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-cert\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.244449 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244264    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42f88ad0-2936-4d56-9fef-a74e38d0c770-crio-socket\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244449 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42f88ad0-2936-4d56-9fef-a74e38d0c770-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244449 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6hs9\" (UniqueName: \"kubernetes.io/projected/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-api-access-q6hs9\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244449 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244449 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244394    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fcb\" (UniqueName: \"kubernetes.io/projected/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-kube-api-access-77fcb\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.244683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244496    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42f88ad0-2936-4d56-9fef-a74e38d0c770-crio-socket\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244650    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f88ad0-2936-4d56-9fef-a74e38d0c770-data-volume\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.244934 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.244914    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.246704 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.246650    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42f88ad0-2936-4d56-9fef-a74e38d0c770-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.246790 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.246724    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-cert\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.256619 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.256597    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:45.301760 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.301730    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6hs9\" (UniqueName: \"kubernetes.io/projected/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-api-access-q6hs9\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.301935 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.301898    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fcb\" (UniqueName: \"kubernetes.io/projected/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-kube-api-access-77fcb\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.302635 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.302610    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:28:45.343345 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.343268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:28:45.536912 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.536876    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:28:45.537091 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.536937    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:28:45.537091 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.537037    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:28:45.566727 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.566694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:45.586441 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586408    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwkt\""
Oct 31 03:28:45.586441 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586408    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 03:28:45.586441 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586438    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 03:28:45.586732 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586548    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 03:28:45.586732 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586566    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 03:28:45.586732 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586572    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-8f2x4\""
Oct 31 03:28:45.586732 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.586605    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lj989\""
Oct 31 03:28:45.631741 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.631658    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8sd9m"]
Oct 31 03:28:45.683442 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.681946    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gzcxv"]
Oct 31 03:28:45.686059 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:45.686033    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42f88ad0_2936_4d56_9fef_a74e38d0c770.slice/crio-b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0 WatchSource:0}: Error finding container b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0: Status 404 returned error can't find the container with id b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0
Oct 31 03:28:45.707190 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.707163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:45.724300 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.724270    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-94ztd"]
Oct 31 03:28:45.727979 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:45.727957    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2ff59e_e274_44b6_968e_49cc6a16f1f3.slice/crio-4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff WatchSource:0}: Error finding container 4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff: Status 404 returned error can't find the container with id 4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff
Oct 31 03:28:45.780503 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.780462    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-94ztd" event={"ID":"aa2ff59e-e274-44b6-968e-49cc6a16f1f3","Type":"ContainerStarted","Data":"4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff"}
Oct 31 03:28:45.782072 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.782043    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzcxv" event={"ID":"42f88ad0-2936-4d56-9fef-a74e38d0c770","Type":"ContainerStarted","Data":"2840b188715c23a310e3a9b2e003a83666e33018b405c3b81f029a7ab9302b32"}
Oct 31 03:28:45.782189 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.782099    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzcxv" event={"ID":"42f88ad0-2936-4d56-9fef-a74e38d0c770","Type":"ContainerStarted","Data":"b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0"}
Oct 31 03:28:45.783197 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.783172    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8sd9m" event={"ID":"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e","Type":"ContainerStarted","Data":"828e9407a613b2f1faaf44b16e24e04a202cbed7639475b743a8bc5a2927df21"}
Oct 31 03:28:45.872839 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.872803    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-dc44455cf-tknzm"]
Oct 31 03:28:45.874928 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.874908    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:45.924067 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.923967    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 03:28:45.924067 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.923990    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 03:28:45.924067 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.924037    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 03:28:45.924067 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.924049    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 03:28:45.924067 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.924062    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 03:28:45.924067 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.924067    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 03:28:45.924376 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.924110    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 03:28:45.924376 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.924110    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wtzlk\""
Oct 31 03:28:45.932824 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:45.932786    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dc44455cf-tknzm"]
Oct 31 03:28:46.051445 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.051636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.051636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.051636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051568    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.051636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051593    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.051636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.051815 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.051738    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152406 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152553 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152553 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152553 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152553 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152553 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.152553 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.152552    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.153085 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.153059    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.153447 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.153428    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.153552 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.153517    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.153607 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.153552    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.154930 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.154912    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.155313 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.155298    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.210712 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.210647    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.485236 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.485131    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:46.894896 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:46.894739    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dc44455cf-tknzm"]
Oct 31 03:28:46.901368 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:46.901338    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3bcb79d_3ca1_49f4_bbe3_19d10b620a45.slice/crio-ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7 WatchSource:0}: Error finding container ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7: Status 404 returned error can't find the container with id ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7
Oct 31 03:28:47.791563 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:47.791516    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzcxv" event={"ID":"42f88ad0-2936-4d56-9fef-a74e38d0c770","Type":"ContainerStarted","Data":"f9db8e6b44a61cf44acca6b5a636ea1b3cfca0091269838c41a2845f297299ff"}
Oct 31 03:28:47.793007 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:47.792980    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc44455cf-tknzm" event={"ID":"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45","Type":"ContainerStarted","Data":"ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7"}
Oct 31 03:28:48.563516 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.563478    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:48.700189 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.700144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:48.798167 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.798127    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8sd9m" event={"ID":"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e","Type":"ContainerStarted","Data":"b7674bc0a9df4540e02943be7e86288795d8d66ab39146a926d86c971d4886af"}
Oct 31 03:28:48.798367 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.798186    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8sd9m" event={"ID":"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e","Type":"ContainerStarted","Data":"8118c5a04d43469be00dd460e2b90b51eda007ce57bfc7ed37d12b7a5d52a4dc"}
Oct 31 03:28:48.798367 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.798322    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:48.838276 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.838187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:48.907961 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.907894    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-8sd9m" podStartSLOduration=2.425681695 podStartE2EDuration="4.907874735s" totalImagesPullingTime="2.48219304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:48.907058008 +0000 UTC m=+52.826219250" watchObservedRunningTime="2026-10-31 03:28:48.907874735 +0000 UTC m=+52.827035979"
Oct 31 03:28:48.954048 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.954008    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"]
Oct 31 03:28:48.956451 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.956429    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:48.979259 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:48.979223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:49.007875 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.007830    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 03:28:49.008379 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.008140    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 03:28:49.008379 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.008224    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 03:28:49.008379 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.008342    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 03:28:49.009647 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.009623    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"]
Oct 31 03:28:49.011044 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.011025    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 03:28:49.011145 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.011099    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-npq57\""
Oct 31 03:28:49.077675 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.077629    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.077675 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.077678    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.077923 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.077700    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ls5m\" (UniqueName: \"kubernetes.io/projected/5cd7aa60-8222-467f-88e2-88cb44ff12a5-kube-api-access-4ls5m\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.077923 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.077742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd7aa60-8222-467f-88e2-88cb44ff12a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.179094 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.178979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.179273 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.179136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.179273 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.179163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ls5m\" (UniqueName: \"kubernetes.io/projected/5cd7aa60-8222-467f-88e2-88cb44ff12a5-kube-api-access-4ls5m\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.179273 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.179255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd7aa60-8222-467f-88e2-88cb44ff12a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.180138 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.180078    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd7aa60-8222-467f-88e2-88cb44ff12a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.183188 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.183162    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.185846 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.185820    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.237939 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.237908    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ls5m\" (UniqueName: \"kubernetes.io/projected/5cd7aa60-8222-467f-88e2-88cb44ff12a5-kube-api-access-4ls5m\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.271641 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.271605    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:28:49.647317 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.647269    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"]
Oct 31 03:28:49.653390 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:49.653352    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd7aa60_8222_467f_88e2_88cb44ff12a5.slice/crio-fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b WatchSource:0}: Error finding container fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b: Status 404 returned error can't find the container with id fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b
Oct 31 03:28:49.804073 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.804032    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzcxv" event={"ID":"42f88ad0-2936-4d56-9fef-a74e38d0c770","Type":"ContainerStarted","Data":"1df34f339f8058cdc719895de43903f72cb188dec9bcfe312891787568ecf50e"}
Oct 31 03:28:49.805425 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.805396    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg" event={"ID":"5cd7aa60-8222-467f-88e2-88cb44ff12a5","Type":"ContainerStarted","Data":"fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b"}
Oct 31 03:28:49.807080 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.807056    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-94ztd" event={"ID":"aa2ff59e-e274-44b6-968e-49cc6a16f1f3","Type":"ContainerStarted","Data":"5ec11ee7d16ec6eff4e1e2d1150465e5abe7c33b85a6d414493c6c96659c7b8e"}
Oct 31 03:28:49.915154 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:49.915102    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gzcxv" podStartSLOduration=2.382324611 podStartE2EDuration="5.91508572s" totalImagesPullingTime="3.532761109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:28:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:49.914018562 +0000 UTC m=+53.833179861" watchObservedRunningTime="2026-10-31 03:28:49.91508572 +0000 UTC m=+53.834247004"
Oct 31 03:28:50.022726 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:50.022665    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-94ztd" podStartSLOduration=2.335266678 podStartE2EDuration="6.022646146s" totalImagesPullingTime="3.687379468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:50.022047579 +0000 UTC m=+53.941208821" watchObservedRunningTime="2026-10-31 03:28:50.022646146 +0000 UTC m=+53.941807389"
Oct 31 03:28:50.814614 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:50.814536    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc44455cf-tknzm" event={"ID":"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45","Type":"ContainerStarted","Data":"47bb757ff7524da3958839b661189e02f3616fecb541e7212868624b8a8102c3"}
Oct 31 03:28:50.926142 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:50.926069    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-dc44455cf-tknzm" podStartSLOduration=2.30992183 podStartE2EDuration="5.926050292s" totalImagesPullingTime="3.616128462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:50.925277395 +0000 UTC m=+54.844438638" watchObservedRunningTime="2026-10-31 03:28:50.926050292 +0000 UTC m=+54.845211535"
Oct 31 03:28:51.819625 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:51.819577    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg" event={"ID":"5cd7aa60-8222-467f-88e2-88cb44ff12a5","Type":"ContainerStarted","Data":"6a1aac509775d83d6dc22c623821de2859207f91b213e17032399dc839e6e1ca"}
Oct 31 03:28:51.820025 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:51.819648    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg" event={"ID":"5cd7aa60-8222-467f-88e2-88cb44ff12a5","Type":"ContainerStarted","Data":"df0c7972d4d16760f9cbba7134afe36493e00e2e267c40abfa208a9ce12db5ad"}
Oct 31 03:28:51.962636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:51.962531    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg" podStartSLOduration=2.021183013 podStartE2EDuration="3.962516544s" totalImagesPullingTime="1.941333531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:51.962183904 +0000 UTC m=+55.881345146" watchObservedRunningTime="2026-10-31 03:28:51.962516544 +0000 UTC m=+55.881677786"
Oct 31 03:28:52.743401 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:52.743370    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:28:53.898591 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:53.898554    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:54.039475 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:54.039436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:54.210101 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:54.210028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:54.807681 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:54.807636    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:54.958111 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:54.958077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:55.225009 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.224901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:55.375477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.375442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:55.424020 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.423983    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"]
Oct 31 03:28:55.426322 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.426305    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.431326 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.431303    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.431449 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.431376    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.431515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.431439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmvtk\" (UniqueName: \"kubernetes.io/projected/0acad573-0b62-497a-9068-546a37bbc5ea-kube-api-access-lmvtk\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.431577 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.431554    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acad573-0b62-497a-9068-546a37bbc5ea-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.475298 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.475195    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 31 03:28:55.475463 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.475447    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fn57l\""
Oct 31 03:28:55.475501 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.475469    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 31 03:28:55.522815 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.522779    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"]
Oct 31 03:28:55.532362 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.532322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.532362 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.532365    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmvtk\" (UniqueName: \"kubernetes.io/projected/0acad573-0b62-497a-9068-546a37bbc5ea-kube-api-access-lmvtk\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.532618 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.532405    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acad573-0b62-497a-9068-546a37bbc5ea-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.532683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.532658    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.533329 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.533304    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acad573-0b62-497a-9068-546a37bbc5ea-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.533729 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.533708    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-q5pft"]
Oct 31 03:28:55.535013 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.534987    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.535095 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.535040    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.536036 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.536020    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.537121 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.537097    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"]
Oct 31 03:28:55.539558 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.539533    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.594032 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.593997    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmvtk\" (UniqueName: \"kubernetes.io/projected/0acad573-0b62-497a-9068-546a37bbc5ea-kube-api-access-lmvtk\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.607892 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.607857    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"]
Oct 31 03:28:55.609681 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.609651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:55.618149 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.618080    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 31 03:28:55.618333 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.618316    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 03:28:55.618412 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.618396    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 03:28:55.629659 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.629635    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 03:28:55.629659 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.629654    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b9rvf\""
Oct 31 03:28:55.629810 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.629661    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-24j69\""
Oct 31 03:28:55.629810 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.629668    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 31 03:28:55.629810 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.629687    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 31 03:28:55.633064 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-wtmp\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633159 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633077    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.633159 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.633278 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633178    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-sys\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633278 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633201    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-tls\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633278 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633253    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-metrics-client-ca\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633381 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633295    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-textfile\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633420 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633389    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.633420 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msp4w\" (UniqueName: \"kubernetes.io/projected/35631c59-2c9c-4672-af14-6f7f6734fc24-kube-api-access-msp4w\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633506 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633442    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-root\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633506 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.633588 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.633588 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.633563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-accelerators-collector-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.634076 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.634059    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfmh5\" (UniqueName: \"kubernetes.io/projected/d0ca45c9-f001-466d-82b0-852e1e486661-kube-api-access-nfmh5\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.634109 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.634089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ca45c9-f001-466d-82b0-852e1e486661-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735281    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735371    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msp4w\" (UniqueName: \"kubernetes.io/projected/35631c59-2c9c-4672-af14-6f7f6734fc24-kube-api-access-msp4w\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-root\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735476    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735541    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-accelerators-collector-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735582    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfmh5\" (UniqueName: \"kubernetes.io/projected/d0ca45c9-f001-466d-82b0-852e1e486661-kube-api-access-nfmh5\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ca45c9-f001-466d-82b0-852e1e486661-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735647    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-wtmp\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735681    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735720    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735747    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-sys\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735769    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-tls\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-metrics-client-ca\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.735835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-textfile\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.738408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.736327    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-textfile\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.739365 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.737057    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.739365 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.737499    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-root\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.739365 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.738129    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.739365 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.738434    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:28:55.739365 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.739036    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-sys\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.739365 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.739137    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-wtmp\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.739656 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.739408    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ca45c9-f001-466d-82b0-852e1e486661-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.740047 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.740016    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-metrics-client-ca\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.740165 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.740131    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-accelerators-collector-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.741538 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.741372    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.741538 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.741492    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.742111 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.742090    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.742393 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.742368    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-tls\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.783719 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.783682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:55.797432 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.797398    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msp4w\" (UniqueName: \"kubernetes.io/projected/35631c59-2c9c-4672-af14-6f7f6734fc24-kube-api-access-msp4w\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.802919 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.802889    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfmh5\" (UniqueName: \"kubernetes.io/projected/d0ca45c9-f001-466d-82b0-852e1e486661-kube-api-access-nfmh5\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.856927 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.856894    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:28:55.861754 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:55.861722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:28:55.868128 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:55.868102    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35631c59_2c9c_4672_af14_6f7f6734fc24.slice/crio-9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1 WatchSource:0}: Error finding container 9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1: Status 404 returned error can't find the container with id 9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1
Oct 31 03:28:56.029231 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.029173    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:56.149960 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.149929    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"]
Oct 31 03:28:56.153141 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:56.153117    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acad573_0b62_497a_9068_546a37bbc5ea.slice/crio-41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51 WatchSource:0}: Error finding container 41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51: Status 404 returned error can't find the container with id 41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51
Oct 31 03:28:56.226907 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.226876    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"]
Oct 31 03:28:56.232899 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:56.232871    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ca45c9_f001_466d_82b0_852e1e486661.slice/crio-4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988 WatchSource:0}: Error finding container 4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988: Status 404 returned error can't find the container with id 4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988
Oct 31 03:28:56.236865 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.236833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:56.486066 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.485998    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:56.486403 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.486383    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:56.490876 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.490856    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:56.837998 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.837964    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7" event={"ID":"d0ca45c9-f001-466d-82b0-852e1e486661","Type":"ContainerStarted","Data":"4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988"}
Oct 31 03:28:56.839125 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.839090    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q5pft" event={"ID":"35631c59-2c9c-4672-af14-6f7f6734fc24","Type":"ContainerStarted","Data":"9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1"}
Oct 31 03:28:56.840863 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.840836    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h" event={"ID":"0acad573-0b62-497a-9068-546a37bbc5ea","Type":"ContainerStarted","Data":"1f42349f846d6be22273684fcbfc8c860edc6382697f98643d5096e3ffdc6a98"}
Oct 31 03:28:56.841001 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.840895    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h" event={"ID":"0acad573-0b62-497a-9068-546a37bbc5ea","Type":"ContainerStarted","Data":"3905b45061a82246e3768c0693ce2d20b3aa4146c00b7aabe246c490adfc4ae6"}
Oct 31 03:28:56.841001 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.840931    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h" event={"ID":"0acad573-0b62-497a-9068-546a37bbc5ea","Type":"ContainerStarted","Data":"41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51"}
Oct 31 03:28:56.844789 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.844767    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:28:56.883294 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:56.883253    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:57.222480 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.222382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:57.711000 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.710943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:57.758555 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.758522    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:28:57.761843 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.761817    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817343    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817343    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817384    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817343    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817394    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817410    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 31 03:28:57.817477 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817444    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 31 03:28:57.818045 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817583    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 31 03:28:57.818045 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.817638    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njklp\""
Oct 31 03:28:57.827604 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.827572    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:28:57.847237 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.847129    2583 generic.go:356] "Generic (PLEG): container finished" podID="35631c59-2c9c-4672-af14-6f7f6734fc24" containerID="292718d56dd78d2a145245d931d3f0b422c8ec50aa7b5b46d54949e0e1c8c3c6" exitCode=0
Oct 31 03:28:57.847403 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.847247    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q5pft" event={"ID":"35631c59-2c9c-4672-af14-6f7f6734fc24","Type":"ContainerDied","Data":"292718d56dd78d2a145245d931d3f0b422c8ec50aa7b5b46d54949e0e1c8c3c6"}
Oct 31 03:28:57.855262 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855364 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855364 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855329    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855364 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855354    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855483 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855525 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855500    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855526    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855569 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855555    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855646 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855646 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855634    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855730 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855674    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855730 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855714    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.855802 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.855744    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.891425 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.891380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:57.921216 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.921169    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 31 03:28:57.956290 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.956255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.956290 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.956297    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.956530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.956343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.956530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.956387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.956530 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.956417    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957308    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957344    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957413    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957525    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.958809 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.958802    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.959374 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.958847    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.959374 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.957343    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.960445 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.959471    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.961511 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.961486    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.961978 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.961945    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.962981 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.962949    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.964093 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.964071    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.965019 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.964977    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.966060 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.966028    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:57.966609 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:57.966590    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:58.017048 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.017010    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:58.080064 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.080036    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8444c575f6-l95rj"]
Oct 31 03:28:58.083216 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.083189    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.083384 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.083360    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:28:58.132563 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.132119    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 31 03:28:58.135851 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.134324    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 31 03:28:58.135851 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.134456    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4m9nhhr1ar1k5\""
Oct 31 03:28:58.135851 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.134525    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 31 03:28:58.136593 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.136565    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 31 03:28:58.145868 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.136685    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 31 03:28:58.158590 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.158496    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dl6f5\""
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159133    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8444c575f6-l95rj"]
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159482    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb58p\" (UniqueName: \"kubernetes.io/projected/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-kube-api-access-rb58p\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159502    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159522    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-metrics-client-ca\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159541    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159671 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159662    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-tmp\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159932 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159766    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159932 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159814    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-grpc-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.159932 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.159856    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.260792 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.260758    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb58p\" (UniqueName: \"kubernetes.io/projected/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-kube-api-access-rb58p\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.260796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.260826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-metrics-client-ca\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.260855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-tmp\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-grpc-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261180    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261678 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261523    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-tmp\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261678 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261593    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-metrics-client-ca\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.261678 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.261631    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.263622 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.263599    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.264006 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.263983    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.264358 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.264341    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-grpc-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.264408 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.264343    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.264478 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.264457    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.265244 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.265227    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.319684 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.319620    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb58p\" (UniqueName: \"kubernetes.io/projected/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-kube-api-access-rb58p\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.418824 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.418792    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:28:58.495552 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.495524    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59c57d5c75-2rvq9"]
Oct 31 03:28:58.498313 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.498298    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.509580 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.509555    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:28:58.512662 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:58.512636    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91095503_a34e_402b_9017_84b2a5379894.slice/crio-65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9 WatchSource:0}: Error finding container 65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9: Status 404 returned error can't find the container with id 65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9
Oct 31 03:28:58.547841 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.547811    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 03:28:58.555525 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.555495    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c57d5c75-2rvq9"]
Oct 31 03:28:58.565690 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565659    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.565837 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565698    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.565837 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565794    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.565963 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.565963 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565903    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.565963 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.565963 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.565942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.566159 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.566037    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.667121 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.667121 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667065    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.667121 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.667121 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667114    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667194    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.667868    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.668472    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.669310    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.669426    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.669899    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.670274    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.672771 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.672751    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.673051 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.673028    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.727836 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.727804    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.790652 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.790621    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8444c575f6-l95rj"]
Oct 31 03:28:58.794537 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:58.794503    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6862e1ee_e114_483c_b37a_1ce7ddeb52aa.slice/crio-e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e WatchSource:0}: Error finding container e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e: Status 404 returned error can't find the container with id e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e
Oct 31 03:28:58.808284 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.808257    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:28:58.809979 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.809953    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:28:58.862120 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.861243    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7" event={"ID":"d0ca45c9-f001-466d-82b0-852e1e486661","Type":"ContainerStarted","Data":"25b4b9b88869d093d75adbc78ed328cf5acdabdfaf4dd5e34b08baa3d1a75d33"}
Oct 31 03:28:58.862120 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.861349    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7" event={"ID":"d0ca45c9-f001-466d-82b0-852e1e486661","Type":"ContainerStarted","Data":"269963e79149c9d32e1cfd33e107c88a4bcee315cccc7f44254c97c0b167e027"}
Oct 31 03:28:58.862120 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.861381    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7" event={"ID":"d0ca45c9-f001-466d-82b0-852e1e486661","Type":"ContainerStarted","Data":"a13293d1108422649ec7133468ef80545874514397f3d7ecd11d1557a1a4281f"}
Oct 31 03:28:58.865356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.865320    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q5pft" event={"ID":"35631c59-2c9c-4672-af14-6f7f6734fc24","Type":"ContainerStarted","Data":"7ec60870697cbbf3eda435e7d7287067a0256a30fd5d1b929eceb7631548b6ef"}
Oct 31 03:28:58.865556 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.865384    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-q5pft" event={"ID":"35631c59-2c9c-4672-af14-6f7f6734fc24","Type":"ContainerStarted","Data":"d38e5b24056fbb22015555bc7ade9f0631a3bab5bfb137c5325ac65430884f88"}
Oct 31 03:28:58.873605 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.873563    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h" event={"ID":"0acad573-0b62-497a-9068-546a37bbc5ea","Type":"ContainerStarted","Data":"16458defa6209f01f5f6167aae0813636688e5c7a07bc3268f05051cdbdfb02d"}
Oct 31 03:28:58.875289 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.875265    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9"}
Oct 31 03:28:58.876637 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:58.876612    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e"}
Oct 31 03:28:59.033068 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.032998    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h" podStartSLOduration=2.259798402 podStartE2EDuration="4.032977468s" totalImagesPullingTime="1.773179066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:59.031526026 +0000 UTC m=+62.950687269" watchObservedRunningTime="2026-10-31 03:28:59.032977468 +0000 UTC m=+62.952138714"
Oct 31 03:28:59.144585 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.144533    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-q5pft" podStartSLOduration=2.544961241 podStartE2EDuration="4.144518642s" totalImagesPullingTime="1.517117291s" totalInitContainerRuntime="82.44011ms" isStatefulPod=true podCreationTimestamp="2026-10-31 03:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:59.143149861 +0000 UTC m=+63.062311103" watchObservedRunningTime="2026-10-31 03:28:59.144518642 +0000 UTC m=+63.063679884"
Oct 31 03:28:59.188162 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.188130    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59c57d5c75-2rvq9"]
Oct 31 03:28:59.191503 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:28:59.191470    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbace1d01_5c41_4521_91b4_7ce780044fbe.slice/crio-350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9 WatchSource:0}: Error finding container 350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9: Status 404 returned error can't find the container with id 350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9
Oct 31 03:28:59.260037 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.259910    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7" podStartSLOduration=2.427696214 podStartE2EDuration="4.259895973s" totalImagesPullingTime="1.832199759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:59.259096833 +0000 UTC m=+63.178258075" watchObservedRunningTime="2026-10-31 03:28:59.259895973 +0000 UTC m=+63.179057214"
Oct 31 03:28:59.540123 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.540089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:59.698812 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.698759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:59.883356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.883188    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c57d5c75-2rvq9" event={"ID":"bace1d01-5c41-4521-91b4-7ce780044fbe","Type":"ContainerStarted","Data":"5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd"}
Oct 31 03:28:59.883356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.883282    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c57d5c75-2rvq9" event={"ID":"bace1d01-5c41-4521-91b4-7ce780044fbe","Type":"ContainerStarted","Data":"350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9"}
Oct 31 03:28:59.898773 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.898727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:28:59.995119 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:28:59.995059    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-59c57d5c75-2rvq9" podStartSLOduration=1.995043872 podStartE2EDuration="1.995043872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:59.993395247 +0000 UTC m=+63.912556541" watchObservedRunningTime="2026-10-31 03:28:59.995043872 +0000 UTC m=+63.914205114"
Oct 31 03:29:00.043420 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.043379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:00.195000 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.194913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:00.391841 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.391810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:00.665142 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.665111    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"]
Oct 31 03:29:00.667475 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.667456    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.716359 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.716315    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 31 03:29:00.716359 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.716354    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 31 03:29:00.716599 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.716372    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 31 03:29:00.716599 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.716402    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z5hqt\""
Oct 31 03:29:00.716599 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.716414    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 03:29:00.716599 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.716483    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c8hm9198qkt62\""
Oct 31 03:29:00.725356 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.725323    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"]
Oct 31 03:29:00.786833 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.786785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.786833 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.786827    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlgqk\" (UniqueName: \"kubernetes.io/projected/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-kube-api-access-tlgqk\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.787047 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.786977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-tls\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.787120 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.787062    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-metrics-server-audit-profiles\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.787120 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.787097    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-client-certs\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.787265 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.787241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-client-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.787312 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.787281    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-audit-log\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.887860 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.887820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.887860 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.887865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlgqk\" (UniqueName: \"kubernetes.io/projected/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-kube-api-access-tlgqk\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888104 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.887912    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-tls\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888104 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.887944    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-metrics-server-audit-profiles\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888104 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.887970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-client-certs\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888104 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.888006    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-client-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888104 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.888043    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-audit-log\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888485 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.888451    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-audit-log\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.888667 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.888644    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.889810 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.889778    2583 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea" exitCode=0
Oct 31 03:29:00.889893 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.889864    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"}
Oct 31 03:29:00.889985 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.889959    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-metrics-server-audit-profiles\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.891766 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.891744    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-tls\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.891837 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.891779    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-client-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.893438 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.893418    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-client-certs\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.946140 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.946057    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlgqk\" (UniqueName: \"kubernetes.io/projected/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-kube-api-access-tlgqk\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:00.979953 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:00.979909    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:01.402704 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:01.402072    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"]
Oct 31 03:29:01.414239 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:29:01.410747    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc6feeb_2db4_4605_b2d8_6220c8d27c9c.slice/crio-944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a WatchSource:0}: Error finding container 944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a: Status 404 returned error can't find the container with id 944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a
Oct 31 03:29:01.895285 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:01.895251    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f" event={"ID":"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c","Type":"ContainerStarted","Data":"944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a"}
Oct 31 03:29:01.897555 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:01.897477    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"eda81a0df66ae469149beaa2cf95de464c3cae29dc6e0ccc11df29c6ff20f4f5"}
Oct 31 03:29:01.897555 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:01.897547    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"a8e0909373bcb88bd540242a088fe9ee3e15ca70836772c18a68c63f9db9ba1c"}
Oct 31 03:29:01.897722 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:01.897587    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"44a719b9628e58e2c049c642c2b16bb5155fab704c806f0f2dea65acb1dd1391"}
Oct 31 03:29:02.403254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.403200    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:02.403437 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.403269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:02.451288 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.451258    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 03:29:02.451453 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.451326    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 03:29:02.455947 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.455883    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:02.456068 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.455962    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:02.504447 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.504408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:02.552184 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.552151    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 03:29:02.602805 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.602773    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 03:29:02.608422 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.608389    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:02.695841 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.695767    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-8f2x4\""
Oct 31 03:29:02.699185 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.698831    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:02.702140 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.702121    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwkt\""
Oct 31 03:29:02.705605 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.705583    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:02.706306 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.706262    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lj989\""
Oct 31 03:29:02.709406 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:02.709174    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:03.089121 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.088323    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xgj98"]
Oct 31 03:29:03.125783 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:29:03.125746    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24753e34_7be9_4a7e_8629_622b22fde606.slice/crio-38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5 WatchSource:0}: Error finding container 38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5: Status 404 returned error can't find the container with id 38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5
Oct 31 03:29:03.135979 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.135953    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7qbtw"]
Oct 31 03:29:03.139599 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.139570    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cwdx2"]
Oct 31 03:29:03.145830 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:29:03.145788    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111ff747_5e37_43eb_8488_ea2ebe3432cc.slice/crio-54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0 WatchSource:0}: Error finding container 54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0: Status 404 returned error can't find the container with id 54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0
Oct 31 03:29:03.146518 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:29:03.146497    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b0fafb_8b2b_4b04_84ed_99e96902b9fd.slice/crio-cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901 WatchSource:0}: Error finding container cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901: Status 404 returned error can't find the container with id cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901
Oct 31 03:29:03.711762 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.711718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:03.850997 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.850959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:03.908097 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.908058    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwdx2" event={"ID":"111ff747-5e37-43eb-8488-ea2ebe3432cc","Type":"ContainerStarted","Data":"54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0"}
Oct 31 03:29:03.911787 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.911749    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"}
Oct 31 03:29:03.911925 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.911832    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"}
Oct 31 03:29:03.911925 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.911874    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"}
Oct 31 03:29:03.911925 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.911917    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"}
Oct 31 03:29:03.912069 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.911954    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"}
Oct 31 03:29:03.912069 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.911990    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerStarted","Data":"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"}
Oct 31 03:29:03.913338 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.913313    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f" event={"ID":"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c","Type":"ContainerStarted","Data":"612c53d366866678a13486eb70e25fd8a9027bef5456c37d919d26dc7d2f75d4"}
Oct 31 03:29:03.914515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.914492    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xgj98" event={"ID":"24753e34-7be9-4a7e-8629-622b22fde606","Type":"ContainerStarted","Data":"38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5"}
Oct 31 03:29:03.918045 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.918019    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"1942058be2297b39096beb1a36ea3d2794ddd48febc23875d03f8bad39b5dc79"}
Oct 31 03:29:03.918132 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.918082    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"218eadbc796a902cfd797ca0cca46f71005e754605f90ea6a03cf0d9e5b34fa6"}
Oct 31 03:29:03.918132 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.918129    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" event={"ID":"6862e1ee-e114-483c-b37a-1ce7ddeb52aa","Type":"ContainerStarted","Data":"58bc10bd0d06d2d6096dbf96c3c510838c0eb59768623422c79c45cd4530b241"}
Oct 31 03:29:03.918253 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.918180    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:03.919358 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:03.919335    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7qbtw" event={"ID":"66b0fafb-8b2b-4b04-84ed-99e96902b9fd","Type":"ContainerStarted","Data":"cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901"}
Oct 31 03:29:04.070807 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.070726    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.232399353 podStartE2EDuration="7.070710938s" totalImagesPullingTime="3.82331806s" totalInitContainerRuntime="14.993525ms" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:29:04.069520657 +0000 UTC m=+67.988681912" watchObservedRunningTime="2026-10-31 03:29:04.070710938 +0000 UTC m=+67.989872179"
Oct 31 03:29:04.180100 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.180035    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f" podStartSLOduration=1.7970274819999998 podStartE2EDuration="4.180016939s" totalImagesPullingTime="2.382989457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:29:04.17960839 +0000 UTC m=+68.098769660" watchObservedRunningTime="2026-10-31 03:29:04.180016939 +0000 UTC m=+68.099178183"
Oct 31 03:29:04.297854 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.295969    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj" podStartSLOduration=2.120506587 podStartE2EDuration="6.295953367s" totalImagesPullingTime="4.17544678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:29:04.29543802 +0000 UTC m=+68.214599270" watchObservedRunningTime="2026-10-31 03:29:04.295953367 +0000 UTC m=+68.215114609"
Oct 31 03:29:04.432996 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.432914    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:04.468758 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.468720    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:29:04.480626 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.480595    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523695    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523771    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523812    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523826    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523966    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.523999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.524534 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.524033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.525235 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.524070    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.525235 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.524105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.525235 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.524139    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.525235 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.524167    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.533865    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534025    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534069    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534157    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534228    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534301    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534366    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534470    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534569    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 03:29:04.534699 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.534627    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 03:29:04.578140 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.578104    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 03:29:04.578337 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.578188    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 03:29:04.578337 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.578265    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ktggl\""
Oct 31 03:29:04.578337 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.578279    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8641tjbr9hbo8\""
Oct 31 03:29:04.621695 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.621666    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 03:29:04.621868 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.621856    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 03:29:04.624282 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.623842    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624689    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624714    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624769    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624794    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624819    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624916    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624950    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.624984    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.625012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.625054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.626270 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.625116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.627069 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.625144    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.627069 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.625168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.627069 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.625190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.627069 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.626621    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.627623 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.627486    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.628029 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.628006    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.629799 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.629228    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.629799 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.629574    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.629900 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.629794    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.630224 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.630190    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.632240 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.630801    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.632240 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.630950    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.637643 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.637594    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.637852 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.637758    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.637852 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.637782    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.638272 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.638245    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.638272 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.638263    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.638636 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.638618    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.639175 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.639113    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.639319 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.639281    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.642802 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.642771    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.643372 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.643348    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.692305 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.692234    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:04.814511 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:04.814164    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:05.310084 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:05.310050    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:29:05.314396 ip-10-0-129-108 kubenswrapper[2583]: W1031 03:29:05.314353    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda647986f_3521_49de_950c_fb705df308ab.slice/crio-15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e WatchSource:0}: Error finding container 15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e: Status 404 returned error can't find the container with id 15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e
Oct 31 03:29:05.935612 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:05.935571    2583 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88" exitCode=0
Oct 31 03:29:05.935764 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:05.935662    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"}
Oct 31 03:29:05.935825 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:05.935772    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e"}
Oct 31 03:29:05.938235 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:05.938196    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwdx2" event={"ID":"111ff747-5e37-43eb-8488-ea2ebe3432cc","Type":"ContainerStarted","Data":"0c0fad2f17457dd5066da1a3af63080e4973dc4a0ecc5a163c91425c15fcc38c"}
Oct 31 03:29:05.938357 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:05.938259    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwdx2" event={"ID":"111ff747-5e37-43eb-8488-ea2ebe3432cc","Type":"ContainerStarted","Data":"b34de0596f58913f5c710bca6d026805726986a82e92f640d6b8725e79f9bd67"}
Oct 31 03:29:06.168578 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:06.168519    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cwdx2" podStartSLOduration=67.385310839 podStartE2EDuration="1m9.168501605s" totalImagesPullingTime="1.783190766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:29:06.167267572 +0000 UTC m=+70.086428897" watchObservedRunningTime="2026-10-31 03:29:06.168501605 +0000 UTC m=+70.087662847"
Oct 31 03:29:06.574170 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:06.574135    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:06.712923 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:06.712886    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:07.288419 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:07.288360    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:07.424598 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:07.424548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:08.337515 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.325215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:08.464056 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.464025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:08.604004 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.603929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:08.808427 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.808390    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:08.808604 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.808458    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:08.814867 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.814844    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:08.952434 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.952351    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7qbtw" event={"ID":"66b0fafb-8b2b-4b04-84ed-99e96902b9fd","Type":"ContainerStarted","Data":"ce72a4105e003a6c94a43097c3c215dcf0d9db77ba4503c5fe1da9df9a8eaacc"}
Oct 31 03:29:08.952795 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.952769    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:08.957621 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:08.957594    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:09.061827 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.061765    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7qbtw" podStartSLOduration=66.72624324 podStartE2EDuration="1m12.061744318s" totalImagesPullingTime="5.335501078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:27:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:29:09.059706875 +0000 UTC m=+72.978868121" watchObservedRunningTime="2026-10-31 03:29:09.061744318 +0000 UTC m=+72.980905560"
Oct 31 03:29:09.200967 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.200928    2583 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dc44455cf-tknzm"]
Oct 31 03:29:09.355323 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.355290    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6db55f7dc6-jpb5c"]
Oct 31 03:29:09.358509 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.358485    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.416114 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.416069    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db55f7dc6-jpb5c"]
Oct 31 03:29:09.481286 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481249    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481469 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481296    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481469 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481469 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481392    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481469 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481424    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481469 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481450    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481679 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.481679 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.481496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582455    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582517    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582634    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.582685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.583254    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.583683 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.583368    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.584163 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.583983    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.584163 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.584135    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.584563 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.584432    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.586254 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.586198    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.590234 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.587024    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.641564 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.641487    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.678088 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.678056    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:09.934299 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.934200    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:09.961134 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:09.961087    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xgj98" event={"ID":"24753e34-7be9-4a7e-8629-622b22fde606","Type":"ContainerStarted","Data":"d1129125b1eac80406176d37fc167fa47bd8f0bc0778d3cface70122b22c450e"}
Oct 31 03:29:10.060094 ip-10-0-129-108 kubenswrapper[2583]: I1031 03:29:10.060051    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6db55f7dc6-jpb5c"]
Oct 31 03:29:10.078863 ip-10-0-129-108 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 03:29:10.088810 ip-10-0-129-108 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 03:29:10.089010 ip-10-0-129-108 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 03:29:10.089064 ip-10-0-129-108 systemd[1]: kubelet.service: Consumed 4.867s CPU time.
Oct 31 03:29:10.099084 ip-10-0-129-108 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 03:29:10.206568 ip-10-0-129-108 kubenswrapper[5906]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:29:10.206568 ip-10-0-129-108 kubenswrapper[5906]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 03:29:10.206568 ip-10-0-129-108 kubenswrapper[5906]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:29:10.206568 ip-10-0-129-108 kubenswrapper[5906]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209137    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209278    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209285    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209290    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209295    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209300    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.209278 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209305    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209309    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209313    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209316    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209319    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209321    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209324    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209327    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209334    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209337    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209340    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209343    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209345    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209354    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209359    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209363    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209366    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209368    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209371    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.209582 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209375    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209378    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209380    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209385    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209389    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209394    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209399    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209404    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209409    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209414    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209419    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209424    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209428    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209433    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209437    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209441    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209444    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209447    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209450    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209452    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.210157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209457    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209464    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209470    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209474    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209477    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209479    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209482    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209486    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209490    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209496    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209500    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209503    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209507    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209509    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209512    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209515    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209517    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209521    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209523    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.210792 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209527    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209531    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209535    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209539    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209544    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209548    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209551    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209555    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209560    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209564    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209569    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209574    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209579    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209583    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209590    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209595    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209599    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209604    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209609    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.211469 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209614    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209618    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209622    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209627    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209631    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209635    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209639    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209644    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209648    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209652    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209655    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209659    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209663    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209667    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209670    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209672    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209677    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209680    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209683    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.212058 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209686    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209689    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209692    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.209695    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210700    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210709    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210712    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210715    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210718    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210722    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210724    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210727    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210730    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210735    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210739    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210744    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210749    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210754    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210759    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210763    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.212664 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210767    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210772    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210777    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210781    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210785    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210789    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210793    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210797    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210800    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210807    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210814    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210819    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210822    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210826    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210830    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210834    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210840    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210844    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210849    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.213347 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210854    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210858    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210863    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210869    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210874    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210879    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210883    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210886    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210889    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210895    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210900    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210904    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210909    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210913    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210918    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210922    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210927    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210931    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210935    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210939    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.213940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210941    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210945    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210947    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210950    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210952    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210955    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210959    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210965    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210970    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210974    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210979    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210983    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.210987    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211000    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211005    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211009    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211019    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211023    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211027    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211031    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.214712 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211035    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211039    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211044    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211049    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211054    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211058    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211064    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211069    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211074    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211079    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211083    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211088    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211092    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211097    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211102    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211106    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211111    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211115    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211120    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.215372 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211124    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211129    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211133    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211138    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211142    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211146    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211150    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211154    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211158    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211162    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211168    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211173    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211903    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211914    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211919    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211923    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211929    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211933    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211938    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211944    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.215993 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211948    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211953    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211958    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211962    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211966    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211971    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211975    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211982    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211989    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.211994    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212000    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212004    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212009    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212013    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212016    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212019    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212022    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212025    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212028    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.216560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212031    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212033    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212036    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212041    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212046    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212050    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212054    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212059    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212064    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212068    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212071    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212074    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212077    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212080    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212083    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212086    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212088    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212091    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212094    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212096    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.217155 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212099    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212102    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212108    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212110    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212113    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212116    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212120    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212124    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212128    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212133    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212138    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212143    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212147    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212151    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212154    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212159    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212164    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212170    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212173    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.217672 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212175    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212178    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212180    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212183    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212186    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212189    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212191    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212194    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212197    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212200    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212218    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212222    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212226    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212230    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212237    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212244    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212248    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212251    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212254    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.218138 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212257    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212260    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212262    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212265    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212268    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212270    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212273    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212276    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212280    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212283    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212286    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212295    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212299    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212303    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212307    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212311    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212316    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212320    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212323    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212326    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.218722 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.212328    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212444    5906 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212457    5906 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212466    5906 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212473    5906 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212480    5906 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212486    5906 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212492    5906 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212500    5906 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212503    5906 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212506    5906 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212510    5906 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212514    5906 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212517    5906 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212520    5906 flags.go:64] FLAG: --cgroup-root=""
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212523    5906 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212526    5906 flags.go:64] FLAG: --client-ca-file=""
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212529    5906 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212532    5906 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212540    5906 flags.go:64] FLAG: --cluster-domain=""
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212544    5906 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212550    5906 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212555    5906 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212560    5906 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 03:29:10.219278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212571    5906 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212574    5906 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212577    5906 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212581    5906 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212585    5906 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212588    5906 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212593    5906 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212596    5906 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212600    5906 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212604    5906 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212607    5906 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212610    5906 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212613    5906 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212617    5906 flags.go:64] FLAG: --enable-server="true"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212620    5906 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212630    5906 flags.go:64] FLAG: --event-burst="100"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212636    5906 flags.go:64] FLAG: --event-qps="50"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212640    5906 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212643    5906 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212647    5906 flags.go:64] FLAG: --eviction-hard=""
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212651    5906 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212654    5906 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212657    5906 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212661    5906 flags.go:64] FLAG: --eviction-soft=""
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212664    5906 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 03:29:10.219951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212667    5906 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212670    5906 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212674    5906 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212677    5906 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212680    5906 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212683    5906 flags.go:64] FLAG: --feature-gates=""
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212688    5906 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212691    5906 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212701    5906 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212706    5906 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212711    5906 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212716    5906 flags.go:64] FLAG: --help="false"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212720    5906 flags.go:64] FLAG: --hostname-override="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212728    5906 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212734    5906 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212739    5906 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212744    5906 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212748    5906 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212753    5906 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212757    5906 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212760    5906 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212763    5906 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212766    5906 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 03:29:10.220685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212771    5906 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212774    5906 flags.go:64] FLAG: --kube-reserved=""
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212778    5906 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212783    5906 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212788    5906 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212793    5906 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212798    5906 flags.go:64] FLAG: --lock-file=""
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212801    5906 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212805    5906 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212808    5906 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212819    5906 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212822    5906 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212825    5906 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212829    5906 flags.go:64] FLAG: --logging-format="text"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212832    5906 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212835    5906 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212838    5906 flags.go:64] FLAG: --manifest-url=""
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212841    5906 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212846    5906 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212850    5906 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212854    5906 flags.go:64] FLAG: --max-pods="110"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212858    5906 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212865    5906 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212870    5906 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212874    5906 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 03:29:10.221664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212878    5906 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212881    5906 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212884    5906 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212892    5906 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212895    5906 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212898    5906 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212901    5906 flags.go:64] FLAG: --pod-cidr=""
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212907    5906 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212912    5906 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212917    5906 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212921    5906 flags.go:64] FLAG: --port="10250"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212924    5906 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212927    5906 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-04ba6f89ae360b258"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212930    5906 flags.go:64] FLAG: --qos-reserved=""
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212934    5906 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212938    5906 flags.go:64] FLAG: --register-node="true"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212943    5906 flags.go:64] FLAG: --register-with-taints=""
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212949    5906 flags.go:64] FLAG: --registry-burst="10"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212954    5906 flags.go:64] FLAG: --registry-qps="5"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212958    5906 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212961    5906 flags.go:64] FLAG: --reserved-memory=""
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212965    5906 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212968    5906 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212971    5906 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212975    5906 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212978    5906 flags.go:64] FLAG: --runonce="false"
Oct 31 03:29:10.222650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212982    5906 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212985    5906 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212988    5906 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212993    5906 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212996    5906 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.212999    5906 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213002    5906 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213005    5906 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213008    5906 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213011    5906 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213014    5906 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213018    5906 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213024    5906 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213030    5906 flags.go:64] FLAG: --system-cgroups=""
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213034    5906 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213040    5906 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213043    5906 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213046    5906 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213051    5906 flags.go:64] FLAG: --tls-min-version=""
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213054    5906 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213057    5906 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213060    5906 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213063    5906 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213066    5906 flags.go:64] FLAG: --v="2"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213071    5906 flags.go:64] FLAG: --version="false"
Oct 31 03:29:10.223434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213075    5906 flags.go:64] FLAG: --vmodule=""
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213079    5906 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213083    5906 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213262    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213268    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213272    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213276    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213281    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213285    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213289    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213293    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213297    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213302    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213306    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213310    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213313    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213316    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213319    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213323    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213327    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.224081 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213335    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213339    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213341    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213344    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213347    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213349    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213353    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213356    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213358    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213361    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213364    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213367    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213369    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213372    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213377    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213381    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213386    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213390    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213395    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213399    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.224654 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213404    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213409    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213413    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213416    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213419    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213422    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213424    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213427    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213430    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213433    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213436    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213438    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213442    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213445    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213447    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213450    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213453    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213456    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213459    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213461    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.225252 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213466    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213470    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213475    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213480    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213484    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213487    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213490    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213493    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213495    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213498    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213500    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213503    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213505    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213508    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213512    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213515    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213518    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213520    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213523    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.225813 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213527    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213531    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213534    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213537    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213542    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213548    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213552    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213557    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213562    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213566    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213571    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213575    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213580    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213584    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213589    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213593    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213598    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213629    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213636    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.226362 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213642    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213647    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213651    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213656    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213660    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213665    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213669    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213672    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213677    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213681    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213684    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.213727    5906 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213788    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213798    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213803    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.227024 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213809    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213813    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213817    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213822    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213828    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213833    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213837    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213840    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213843    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213845    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213848    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213851    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213854    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213856    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213859    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213863    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213867    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213874    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213881    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.227658 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213885    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213891    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213896    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213900    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213905    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213909    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213930    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213936    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213942    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213948    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213954    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213960    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213964    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213969    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213973    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213977    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213981    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213987    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213991    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213996    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.228316 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.213999    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214001    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214004    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214007    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214010    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214012    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214015    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214018    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214021    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214023    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214026    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214029    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214031    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214035    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214040    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214044    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214048    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214050    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214055    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214057    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.228872 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214062    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214065    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214068    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214071    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214076    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214080    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214084    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214088    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214092    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214098    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214102    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214106    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214110    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214115    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214119    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214123    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214127    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214129    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214134    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214139    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.229500 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214144    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214148    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214152    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214157    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214162    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214167    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214172    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214178    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214184    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214188    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214194    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214198    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214214    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214219    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214224    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214228    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214233    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214236    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214239    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214241    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.230173 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214244    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214248    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214251    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.214255    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.218892    5906 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.218911    5906 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.218931    5906 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.218936    5906 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219027    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219035    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219040    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219044    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219047    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219050    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219053    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219056    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219059    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219062    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219064    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.231012 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219067    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219071    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219074    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219076    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219079    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219082    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219085    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219088    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219091    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219093    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219096    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219099    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219101    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219104    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219106    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219109    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219112    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219114    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219118    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219122    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.231691 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219124    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219127    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219132    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219136    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219141    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219145    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219150    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219154    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219157    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219160    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219162    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219167    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219172    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219176    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219181    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219185    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219190    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219193    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219196    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219198    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.232307 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219201    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219216    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219219    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219221    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219224    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219227    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219230    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219233    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219238    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219243    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219247    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219252    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219255    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219258    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219260    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219263    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219266    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219269    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219272    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.232967 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219274    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219277    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219280    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219285    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219289    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219293    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219296    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219298    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219301    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219304    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219307    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219310    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219313    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219316    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219321    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219326    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219330    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219336    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219341    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.233531 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219346    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219350    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219354    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219359    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219362    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219365    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219368    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219371    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219375    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219377    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219380    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219383    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219386    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219388    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219391    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219395    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.234131 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219399    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.219434    5906 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219611    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219617    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219620    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219623    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219626    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219629    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219632    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219634    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219638    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219640    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219643    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219646    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219649    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219652    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.234704 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219654    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219657    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219660    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219663    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219667    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219672    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219677    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219682    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219687    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219691    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219696    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219700    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219704    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219707    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219710    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219712    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219715    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219717    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219720    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219723    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.235242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219726    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219728    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219731    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219734    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219736    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219739    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219742    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219745    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219748    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219751    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219753    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219756    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219759    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219762    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219765    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219769    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219773    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219778    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219782    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219785    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.235856 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219789    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219792    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219795    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219798    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219800    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219804    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219808    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219812    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219817    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219820    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219822    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219825    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219827    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219830    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219832    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219835    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219837    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219840    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219843    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219846    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.236578 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219848    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219851    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219853    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219856    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219858    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219861    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219864    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219870    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219874    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219878    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219882    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219888    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219893    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219898    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219900    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219903    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219906    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219909    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219911    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.237262 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219914    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219916    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219919    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219921    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219924    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219926    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219929    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219931    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219934    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219937    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219940    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219942    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.219945    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.219970    5906 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220011    5906 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:29:10.237953 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220016    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220019    5906 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220022    5906 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220025    5906 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220028    5906 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220032    5906 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220037    5906 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220042    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220047    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220051    5906 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220055    5906 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220060    5906 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220065    5906 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220070    5906 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220074    5906 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220079    5906 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220084    5906 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220089    5906 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220093    5906 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220096    5906 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:29:10.238529 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220099    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220102    5906 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220105    5906 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220108    5906 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220110    5906 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220113    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220116    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220119    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220122    5906 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220125    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220127    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220130    5906 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220133    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220135    5906 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220138    5906 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220143    5906 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220147    5906 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220151    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220155    5906 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220158    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:29:10.239354 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220162    5906 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220167    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220171    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220176    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220181    5906 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220187    5906 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220192    5906 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220196    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220201    5906 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220215    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220219    5906 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220223    5906 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220228    5906 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220232    5906 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220236    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220239    5906 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220242    5906 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220248    5906 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220250    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220254    5906 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:29:10.240055 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220258    5906 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220263    5906 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220267    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220270    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220275    5906 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220280    5906 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220284    5906 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220288    5906 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220291    5906 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220294    5906 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220297    5906 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220299    5906 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220302    5906 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220305    5906 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220308    5906 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220310    5906 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220313    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220315    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220319    5906 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:29:10.240904 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220322    5906 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220325    5906 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220328    5906 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220333    5906 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220337    5906 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220341    5906 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220345    5906 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220348    5906 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220350    5906 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220353    5906 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220358    5906 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220362    5906 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220367    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220371    5906 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220375    5906 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220380    5906 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220386    5906 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220394    5906 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220399    5906 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:29:10.241652 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220404    5906 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220408    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220412    5906 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220417    5906 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220421    5906 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220425    5906 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:29:10.220428    5906 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.220500    5906 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.224252    5906 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.224314    5906 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.224533    5906 server.go:1014] "Starting client certificate rotation"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.224669    5906 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.224718    5906 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 03:22:57 +0000 UTC" deadline="2028-06-29 00:20:38.857264268 +0000 UTC"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.224759    5906 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14564h51m28.632507983s"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.225092    5906 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.226442    5906 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:29:10.242336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.227796    5906 log.go:25] "Validated CRI v1 runtime API"
Oct 31 03:29:10.242958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.230340    5906 log.go:25] "Validated CRI v1 image API"
Oct 31 03:29:10.242958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.231230    5906 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 03:29:10.242958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.234144    5906 fs.go:135] Filesystem UUIDs: map[6d4dc3bf-8bf1-4e56-b097-7cc9ec6dfdb0:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 815a233a-6e66-4944-b74b-46e206fc5792:/dev/nvme0n1p3]
Oct 31 03:29:10.243126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.234166    5906 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e/userdata/shm major:0 minor:485 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9/userdata/shm major:0 minor:372 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5/userdata/shm major:0 minor:409 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4bd18deee211dd6dc6597de097aa941e0501ad95d9666191967976ef8e3511c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4bd18deee211dd6dc6597de097aa941e0501ad95d9666191967976ef8e3511c5/userdata/shm major:0 minor:509 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0/userdata/shm major:0 minor:411 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9/userdata/shm major:0 minor:337 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/828e9407a613b2f1faaf44b16e24e04a202cbed7639475b743a8bc5a2927df21/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/828e9407a613b2f1faaf44b16e24e04a202cbed7639475b743a8bc5a2927df21/userdata/shm major:0 minor:203 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1/userdata/shm major:0 minor:292 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a/userdata/shm major:0 minor:386 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901/userdata/shm major:0 minor:412 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e/userdata/shm major:0 minor:363 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b/userdata/shm major:0 minor:249 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~projected/kube-api-access-lmvtk:{mountpoint:/var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~projected/kube-api-access-lmvtk major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/111ff747-5e37-43eb-8488-ea2ebe3432cc/volumes/kubernetes.io~projected/kube-api-access-wpm6t:{mountpoint:/var/lib/kubelet/pods/111ff747-5e37-43eb-8488-ea2ebe3432cc/volumes/kubernetes.io~projected/kube-api-access-wpm6t major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/111ff747-5e37-43eb-8488-ea2ebe3432cc/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/111ff747-5e37-43eb-8488-ea2ebe3432cc/volumes/kubernetes.io~secret/metrics-certs major:0 minor:403 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e/volumes/kubernetes.io~projected/kube-api-access-l95pf:{mountpoint:/var/lib/kubelet/pods/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e/volumes/kubernetes.io~projected/kube-api-access-l95pf major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2467cba4-f180-4a23-bb6a-5a67c8e86df4/volumes/kubernetes.io~projected/kube-api-access-sthkd:{mountpoint:/var/lib/kubelet/pods/2467cba4-f180-4a23-bb6a-5a67c8e86df4/volumes/kubernetes.io~projected/kube-api-access-sthkd major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24753e34-7be9-4a7e-8629-622b22fde606/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/24753e34-7be9-4a7e-8629-622b22fde606/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:407 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~projected/kube-api-access-msp4w:{mountpoint:/var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~projected/kube-api-access-msp4w major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~projected/kube-api-access-49jch:{mountpoint:/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~projected/kube-api-access-49jch major:0 minor:508 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:507 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:502 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42f88ad0-2936-4d56-9fef-a74e38d0c770/volumes/kubernetes.io~projected/kube-api-access-q6hs9:{mountpoint:/var/lib/kubelet/pods/42f88ad0-2936-4d56-9fef-a74e38d0c770/volumes/kubernetes.io~projected/kube-api-access-q6hs9 major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42f88ad0-2936-4d56-9fef-a74e38d0c770/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/42f88ad0-2936-4d56-9fef-a74e38d0c770/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~projected/kube-api-access-tlgqk:{mountpoint:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~projected/kube-api-access-tlgqk major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:378 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~projected/kube-api-access-4ls5m:{mountpoint:/var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~projected/kube-api-access-4ls5m major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:247 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66b0fafb-8b2b-4b04-84ed-99e96902b9fd/volumes/kubernetes.io~projected/kube-api-access-98nsz:{mountpoint:/var/lib/kubelet/pods/66b0fafb-8b2b-4b04-84ed-99e96902b9fd/volumes/kubernetes.io~projected/kube-api-access-98nsz major:0 minor:408 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~projected/kube-api-access-rb58p:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~projected/kube-api-access-rb58p major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:359 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:353 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a7862a2-9e10-46d0-b7e9-929eb6158d3d/volumes/kubernetes.io~projected/kube-api-access-7gnm2:{mountpoint:/var/lib/kubelet/pods/6a7862a2-9e10-46d0-b7e9-929eb6158d3d/volumes/kubernetes.io~projected/kube-api-access-7gnm2 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/762debf9-cb16-4dc7-9923-c345e031f290/volumes/kubernetes.io~projected/kube-api-access-wbdbd:{mountpoint:/var/lib/kubelet/pods/762debf9-cb16-4dc7-9923-c345e031f290/volumes/kubernetes.io~projected/kube-api-access-wbdbd major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volumes/kubernetes.io~projected/kube-api-access-x497z:{mountpoint:/var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volumes/kubernetes.io~projected/kube-api-access-x497z major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/alertmanager/9 major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/config-reloader/6 major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~empty-dir/config-out major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~projected/kube-api-access-lvrt8:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~projected/kube-api-access-lvrt8 major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~projected/tls-assets major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/config-volume major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/web-config major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/981be9db-6718-42fc-9076-8262bc218c66/volumes/kubernetes.io~projected/kube-api-access-wvntl:{mountpoint:/var/lib/kubelet/pods/981be9db-6718-42fc-9076-8262bc218c66/volumes/kubernetes.io~projected/kube-api-access-wvntl major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9987cbad-f4ba-43be-91f7-f840a6fe0411/volumes/kubernetes.io~projected/kube-api-access-5kpwr:{mountpoint:/var/lib/kubelet/pods/9987cbad-f4ba-43be-91f7-f840a6fe0411/volumes/kubernetes.io~projected/kube-api-access-5kpwr major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:476 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~empty-dir/config-out major:0 minor:481 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~projected/kube-api-access-vs47r:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~projected/kube-api-access-vs47r major:0 minor:484 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~projected/tls-assets major:0 minor:474 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/config major:0 minor:469 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:483 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:482 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:480 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:478 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:475 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:477 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:479 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/web-config major:0 minor:476 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa2ff59e-e274-44b6-968e-49cc6a16f1f3/volumes/kubernetes.io~projected/kube-api-access-77fcb:{mountpoint:/var/lib/kubelet/pods/aa2ff59e-e274-44b6-968e-49cc6a16f1f3/volumes/kubernetes.io~projected/kube-api-access-77fcb major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa2ff59e-e274-44b6-968e-49cc6a16f1f3/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/aa2ff59e-e274-44b6-968e-49cc6a16f1f3/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~projected/kube-api-access-p5rd4:{mountpoint:/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~projected/kube-api-access-p5rd4 major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c04cb90b-5446-4325-a368-a54295d49f33/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/c04cb90b-5446-4325-a368-a54295d49f33/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~projected/kube-api-access-269hl:{mountpoint:/var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~projected/kube-api-access-269hl major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~projected/kube-api-access-nfmh5:{mountpoint:/var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~projected/kube-api-access-nfmh5 major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~projected/kube-api-access-kdwf5:{mountpoint:/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~projected/kube-api-access-kdwf5 major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887/volumes/kubernetes.io~projected/kube-api-access-xgq7n:{mountpoint:/var/lib/kubelet/pods/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887/volumes/kubernetes.io~projected/kube-api-access-xgq7n major:0 minor:63 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/682121e0d97245724280603540b28b639def16ce7e2c08336e2a7a9fc5ba4c6e/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/f184b5de9bc3d460dfbf45643a8bb03449c40db68a9d9ab00a59f1ad9bec4477/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/bc97afceaeccfe529afa46b9ba20056214050e7d39b8c4720d94f6bfa140d58d/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/92bb78d551774d42b7a8f7f39541f625e55182d7351bf1fe0a4fc6215dd23433/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/e1bfc3e2b173fb805b27cf8ef041d2491cb7a4ec1ba59fccb78693ef0f76a7ee/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/c555d6cd9dc4cf29287092daed8386c9cfcfc2e1298d8a41a0bc038bd0d3615a/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/39bc7127665f523eb14d3a3f6f3e8603d67670c75ee7d1bae8d00dcb2a357605/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/6f9f99b3c4612404498baca40974932fdc62847f8bd99d21b4bd832e2c3a20c1/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/ab97995e6ec1754f396e47ab38afb5732078ae7259ae22feaedb9cf14ba98548/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/2b807b49cd0bf763666e66df3539e6e02f90a5bbc42c75a87bffedcd2b1e77c8/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/522c23affd58c5b2b6ce95702099d348440e50ed5fc6cfb9a9d38157b730594e/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/3a506abd350b9fba422448b37d4670a4215cd656a13edeb1687fbc7b40924ee5/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/cf1137dc0f33815c17f1cb10299285c54e1c7f61af0613bdb4e1608e33caf09e/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/222bca59aac5e3a54de75e824a57c770a7e4963c0a4d50d112c2586a8287e160/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/261243ff35e1da08eebe830579cd8242b7b34618c46c66b0cb3fdd3b7a9964f3/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/f3554d8c1608b83d5a0fdd5efffbc8f4477b44d887f762b04409abe3c1c3e0a3/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/78c6db60121e88ef12c05dc7bfe1c41b71d1f572fef62d9369744acc2b38dc74/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/9bd8982760f9dad21742065532b20dbdded9f901819ad9cc9ebfaa922b8ab31c/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/8bcb3ad8f428498820fe5fb3bfd74553060af1902dcf71c784405c82bcca5181/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/07cff59ae245d4dcfd351338c3602366a7311dc1ec4a8bb8ee5632bd752b272f/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/33fe86774d55492c757f6cbffdd2b955e6a912a35671493e2650c3de5069426e/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/2d1dd3c46d18ec8155ac17a509a3a10f250e8c94db460032f6a1b68d5b7efa97/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/4739e6dc5c04ffeb0268a4232924955e3c85aeab87f7ef78d2ebe0a681c8a3e1/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/36121fb6030430e2bddbb0ea86ff0846293a0cc715c51b3a789c5598e72316f6/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/57609eef69e39ebaacd18b749a6b48a968d74a2fdbfc29668b3250acc56b39cb/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/559d281c72af7c41cb525c65b04ecc01daa9228f3d59390289cd115ccad3d258/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/9275cfd2323878c7cafe53ff3bb4edd5d1458a4acd92f530124139a43a04cd89/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/374acfb7f83812ba117402468ab637a022e334f7984155e38342ad7c10edae1c/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/40c46481e83cf157e0542c40842e0ebfdb3aa14de68ef39d1d52f071818dca0d/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/43b5b65dc496a4b8259b1e78a467fe7b313af022125e097e50d0220fa44cae81/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/5988816ac23b7fc8faa5844b4f5a0b292b033ca9b2f51033bf68e150447a061c/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/1185cc109a4b7b0a687c2faf5bfe39e1c456eb66d5ffd3624e52414c16553257/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/57b871a1b03e025ab2e2bd24e280c4a641b54b9986c176093889b848cbb88efe/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/4e17db7e7dbb3fedfde1e0bb177e05f236d0c9f46def68449a0331de035fb64e/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/497655e2fa3a3688be199e897eb23a0162204e448fc0ed241a4529a18bbeefe9/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/e9213abd43908bfc226e74a2fb04e0728eb01db12593e7ba4f069a52809b8b56/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/86e0edf3126ca28ef4eb698156385da9f4a67bf17de5764623dfd4a8faf6bdc0/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/5a217e40efbaf9052653a38bd2c50816f12564819ad1c2786136ff3cf97c0c2c/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/bae6a00b7c852c0daa4eb554a91ec8ec443f9d9b3f39693c34c51751c255f3b0/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/6948e001f23749814f18b70cb6abe7e1584b887d87049654f7c07fdb91dab1da/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/cf635863699de38a7cb8d8c9b10414f10775fe08ecf146a74601b40610e67b7e/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/b7e1e8ba893b5387d25301b1262b40e55f23b3cd37adab9a77053bcfef64927a/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/02362069bcb70193606877ad76292495dc6ee6f0938c2aff0cccc10b43803016/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/ffb57317b3da3f8dfedd5e31dfcd538468003a5228a20667d8f55e39e4be7fef/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-339:{mountpoint:/var/lib/containers/storage/overlay/aa8c0db6041e83ea0f8737e4cb968e9307ddd9ac4fa83e36cb7edfb92ece5262/merged major:0 minor:339 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/b60bfbd91dbfedcff4bc63617a7f4e89e58ebd99a5cd7b355fe954bd5c5d3997/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/8039e1d83967a2fd8f6780edaa34679123529675d94f0c15c23549ee14afc73f/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/4ac5000e37249c3226a5ad22eb32bc7e32d75c321a54e4a09fdd7f04071b3bde/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/98f517a5af0172e1603ac6596d288b46ea9d49ad4d4489687282a8eb38a27dbb/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/3d256dfae1eed332a4c7c592c0230c90a96aa25684580766e5310d776169a485/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/30cd29a85f1f133b8ace8528568bdf4e814af0a7e58084b935c29c3837caeaff/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-390:{mountpoint:/var/lib/containers/storage/overlay/92cc5519dc0448af7b8fb983ea09ab5942bd8d06d29ac2f57ca12dd6e1bc2870/merged major:0 minor:390 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/03c999cb1daf1c540948467ccd01440088dd09f035e9bc0bab9e241d0724a170/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/cf84c59022145ee306153efdb42a8cbff441bde6f7535749d0735d2d89cd6fc6/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/45d719a60aff8aff369715411d4e814f419e76811035a89054e5c9d9c69279f3/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-417:{mountpoint:/var/lib/containers/storage/overlay/6bb7b947dd6a0799ccddda01be1fcbd36b7b3719c63bc450cbcae5d2dd4814dd/merged major:0 minor:417 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/0bb4c4b86c07c02f7b5b1cc008336452347fd72a3a0b65936118237a10b12b76/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/f6a453b686a8e2d69af429a97963b27dc3c675c6c31658ba20152d9587761180/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/570c5b251b93016417d8c8d33bdf7081bd55d1a2c89c26691b6dc9fc4e8f679f/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-432:{mountpoint:/var/lib/containers/storage/overlay/e3035ddd1a5bacf336359459cda4b26cb1e85bf9cc37079837774c03480e94a3/merged major:0 minor:432 fsType:overlay blockSize:0} overlay_0-434:{mountpoint:/var/lib/containers/storage/overlay/05f175d144cf76a1a578cb09181a047abec3f11f965ae8b59766dd7c681b893a/merged major:0 minor:434 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/4f7da33030c59425f3b619ab905ea24313dab3793935b3b95b69e3fed08c3872/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-440:{mountpoint:/var/lib/containers/storage/overlay/c5ce3360aa9e000c00af484acc99e240abe0542916d8a98663c03d04a3f8539d/merged major:0 minor:440 fsType:overlay blockSize:0} overlay_0-453:{mountpoint:/var/lib/containers/storage/overlay/a18e157412d07d8bc69301d6c96b05d4f1e0216779c3480f6cb9220b026dd047/merged major:0 minor:453 fsType:overlay blockSize:0} overlay_0-455:{mountpoint:/var/lib/containers/storage/overlay/b81c01a92e0cf8d84f746b2d8190d3eae461af02d200bbdf822465d50c9bd1f3/merged major:0 minor:455 fsType:overlay blockSize:0} overlay_0-46:{mountpoint:/var/lib/containers/storage/overlay/2ad92ee6378382ecaf1191a246c256361d08874c98cffcbbff9333197f5dc961/merged major:0 minor:46 fsType:overlay blockSize:0} overlay_0-460:{mountpoint:/var/lib/containers/storage/overlay/ea6d635108b5b7a7498adec30bba115552320ca03dfd17018102e4fd63ef4aa3/merged major:0 minor:460 fsType:overlay blockSize:0} overlay_0-463:{mountpoint:/var/lib/containers/storage/overlay/a44faf7a61e3f28d3f888b9934ad95302b7eac37b727200116b35e735a64c0ef/merged major:0 minor:463 fsType:overlay blockSize:0} overlay_0-487:{mountpoint:/var/lib/containers/storage/overlay/fafdc6f254d38bae0d588b9be1d8d190aa293a2276875e38036846c03e1b8c46/merged major:0 minor:487 fsType:overlay blockSize:0} overlay_0-489:{mountpoint:/var/lib/containers/storage/overlay/94c3b4c35bb4da6e844281d7b781f81bcbf3fb612fac216b6f7f6ef3f524f303/merged major:0 minor:489 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/1acb08a405f6e4cfaa9e763daafafab070113d7c87fbb3441fa0158622c1abeb/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-495:{mountpoint:/var/lib/containers/storage/overlay/dab78178fb1f5638b55ebf2ac26e903eb07c2bab6ae33d8f599e486e3524b903/merged major:0 minor:495 fsType:overlay blockSize:0} overlay_0-500:{mountpoint:/var/lib/containers/storage/overlay/4c70a9c3b41bbf8eb59019f5040b85c2a0a4351487aa917ebfa97a8ffeefe3e1/merged major:0 minor:500 fsType:overlay blockSize:0} overlay_0-511:{mountpoint:/var/lib/containers/storage/overlay/78f1901754c1a2dec09a044ff08fe8f8a83e3a07953fd5c3d6d08317224661ae/merged major:0 minor:511 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/0e37e10b15af8b40fa00cb6692dadf96bf09e8d9ff3e8693374040f019a1a46d/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/982dbe732b8507596de4fb9455e43a90ea3e5093fce30196341f33a348132cfa/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/d795f72ff9fabc9484fe16b2f1d0ffef0a6e61a65d35578a65451aaebbff24a7/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/d51274fc031bea25a3650151bb39a0252e19b23758c64081cf915c567dc92eab/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/ec1a9c5a775987a6f2ed7f32416a4cf8a48069e88b8a4c8d0784d0f06e342042/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/ee3e2bca788a5d1dc212cfba1c97c93990e05b6ca6983817b53395d79e80505b/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/c6a2b81e1d53ed04fcbfb69c3891791878596e08f2bb2481ffbffd0f55a4c07b/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/bb009534611251e910d00f6f2be8bfe2951027cd98e5d7d3a37efc3325810e1c/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/4ea725b646d258dc50343cddec0e10d1828ed2b672b4cb57a84213ba3dcd85fd/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 03:29:10.246869 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.245525    5906 manager.go:217] Machine: {Timestamp:2026-10-31 03:29:10.244897302 +0000 UTC m=+0.078757695 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100106 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec286c79c75a8890acfa42712a9ec5bc SystemUUID:ec286c79-c75a-8890-acfa-42712a9ec5bc BootID:8b726ae0-1adf-4b81-aba1-6e86d6ebb978 Filesystems:[{Device:/run/containers/storage/overlay-containers/27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5d31cbe8291f19a0bc47632b441b0ef44e2de4ff00f5a3dd11095cd315f4b850/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9131c9dac8a5a7d6d4e22e1bffb6ea52c9cf944091d0f11e0b5099436ff4f8e1/userdata/shm DeviceMajor:0 DeviceMinor:292 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4c30785328c0669fcbbef2d00535099d711da6c240700f56e67ab55a71429988/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:361 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c04cb90b-5446-4325-a368-a54295d49f33/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42f88ad0-2936-4d56-9fef-a74e38d0c770/volumes/kubernetes.io~projected/kube-api-access-q6hs9 DeviceMajor:0 DeviceMinor:205 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:322 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:475 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:217 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:323 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/111ff747-5e37-43eb-8488-ea2ebe3432cc/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:403 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-440 DeviceMajor:0 DeviceMinor:440 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-495 DeviceMajor:0 DeviceMinor:495 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/981be9db-6718-42fc-9076-8262bc218c66/volumes/kubernetes.io~projected/kube-api-access-wvntl DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/762debf9-cb16-4dc7-9923-c345e031f290/volumes/kubernetes.io~projected/kube-api-access-wbdbd DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:278 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-339 DeviceMajor:0 DeviceMinor:339 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-46 DeviceMajor:0 DeviceMinor:46 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:282 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:284 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:320 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:329 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:478 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:358 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:483 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:285 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:286 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-417 DeviceMajor:0 DeviceMinor:417 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/35631c59-2c9c-4672-af14-6f7f6734fc24/volumes/kubernetes.io~projected/kube-api-access-msp4w DeviceMajor:0 DeviceMinor:290 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:324 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-453 DeviceMajor:0 DeviceMinor:453 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/38908d5cf1a64d775018935eb1341053cec2aa3620308ca4dbce71a51aecb4d5/userdata/shm DeviceMajor:0 DeviceMinor:409 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-460 DeviceMajor:0 DeviceMinor:460 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:476 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e/volumes/kubernetes.io~projected/kube-api-access-l95pf DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:353 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:469 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:477 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:480 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:224 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:482 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-500 DeviceMajor:0 DeviceMinor:500 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:378 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:479 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-511 DeviceMajor:0 DeviceMinor:511 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42f88ad0-2936-4d56-9fef-a74e38d0c770/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b827d3d30e2230a5ad59142a6bb029bbd732d9f3081d7f4e6a5f7a18a34721d0/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:247 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:368 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6a7862a2-9e10-46d0-b7e9-929eb6158d3d/volumes/kubernetes.io~projected/kube-api-access-7gnm2 DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:287 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:325 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:357 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2467cba4-f180-4a23-bb6a-5a67c8e86df4/volumes/kubernetes.io~projected/kube-api-access-sthkd DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f0c1f8d1b7e7b7464fd8f19725b8a1049d5eeb25675ddb0cfd204cbf78cc423/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/98faa51a33ae3c311fa1ba3e95912239b7777e455720d323c6152a244a782739/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:320 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:474 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e/userdata/shm DeviceMajor:0 DeviceMinor:485 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:476 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~projected/kube-api-access-269hl DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa2ff59e-e274-44b6-968e-49cc6a16f1f3/volumes/kubernetes.io~projected/kube-api-access-77fcb DeviceMajor:0 DeviceMinor:206 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:243 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:327 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:359 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~projected/kube-api-access-tlgqk DeviceMajor:0 DeviceMinor:385 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-390 DeviceMajor:0 DeviceMinor:390 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24753e34-7be9-4a7e-8629-622b22fde606/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:407 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:502 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5794049df4f351824dc54866291f31f73c79d6aea51d6c715a423b3b9c2c9f18/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9/userdata/shm DeviceMajor:0 DeviceMinor:337 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/944ee0c33a721dc2f7d157a120169193628b7d76a2125ea9f089972a7f4d753a/userdata/shm DeviceMajor:0 DeviceMinor:386 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54197d9403fcd657fa39cfc30040ab1e015314870ba1ea074c290d0ed66d7bd0/userdata/shm DeviceMajor:0 DeviceMinor:411 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-432 DeviceMajor:0 DeviceMinor:432 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/41a737dbb888e66bf6c5bea3980403e2e868fdeea089dc844b7f6e7d4aa47e51/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-489 DeviceMajor:0 DeviceMinor:489 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887/volumes/kubernetes.io~projected/kube-api-access-xgq7n DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d0ca45c9-f001-466d-82b0-852e1e486661/volumes/kubernetes.io~projected/kube-api-access-nfmh5 DeviceMajor:0 DeviceMinor:291 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc0f7ed2aba9170d0607890789b3fa5ce82d36f7e31850f232e4a9ff445cc901/userdata/shm DeviceMajor:0 DeviceMinor:412 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:320 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~projected/kube-api-access-49jch DeviceMajor:0 DeviceMinor:508 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e11ef808e59e662847f0ef998e8e8f1185752a6376ff789b3c09dfe4f777298e/userdata/shm DeviceMajor:0 DeviceMinor:363 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:367 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-487 DeviceMajor:0 DeviceMinor:487 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes/kubernetes.io~projected/kube-api-access-kdwf5 DeviceMajor:0 DeviceMinor:225 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/5cd7aa60-8222-467f-88e2-88cb44ff12a5/volumes/kubernetes.io~projected/kube-api-access-4ls5m DeviceMajor:0 DeviceMinor:248 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:383 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-455 DeviceMajor:0 DeviceMinor:455 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-463 DeviceMajor:0 DeviceMinor:463 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa2ff59e-e274-44b6-968e-49cc6a16f1f3/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:360 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9/userdata/shm DeviceMajor:0 DeviceMinor:372 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:320 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-434 DeviceMajor:0 DeviceMinor:434 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~projected/kube-api-access-vs47r DeviceMajor:0 DeviceMinor:484 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc3764b4471e874167b878721b61a2a359a488d15aecd3bddf117ada4d4113c6/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0acad573-0b62-497a-9068-546a37bbc5ea/volumes/kubernetes.io~projected/kube-api-access-lmvtk DeviceMajor:0 DeviceMinor:283 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:326 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:507 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3f7116c0e9344067e78e695605326988d02ee101b43546b19ce7c80ce2b403c7/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~projected/kube-api-access-lvrt8 DeviceMajor:0 DeviceMinor:333 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/828e9407a613b2f1faaf44b16e24e04a202cbed7639475b743a8bc5a2927df21/userdata/shm DeviceMajor:0 DeviceMinor:203 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4958e8f74e84f4f36bc8c07a65e8cffeca5d462bb5b50eb1faae8b93e0a429ff/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:481 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/9987cbad-f4ba-43be-91f7-f840a6fe0411/volumes/kubernetes.io~projected/kube-api-access-5kpwr DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fef341aef37e1c37062dc8a98f641bdad5d54ce276196a3c3b880cafe969af8b/userdata/shm DeviceMajor:0 DeviceMinor:249 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:384 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4bd18deee211dd6dc6597de097aa941e0501ad95d9666191967976ef8e3511c5/userdata/shm DeviceMajor:0 DeviceMinor:509 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/111ff747-5e37-43eb-8488-ea2ebe3432cc/volumes/kubernetes.io~projected/kube-api-access-wpm6t DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0680389d8f2020aaa031b1b670f784d09b0c80cebcf3c2c74220e4a4d21f89b5/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:326 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes/kubernetes.io~projected/kube-api-access-p5rd4 DeviceMajor:0 DeviceMinor:369 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/34adc00643107eb9811c68c6033633cc9e83fd280420f942f9a295e26d44bc62/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6862e1ee-e114-483c-b37a-1ce7ddeb52aa/volumes/kubernetes.io~projected/kube-api-access-rb58p DeviceMajor:0 DeviceMinor:362 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/66b0fafb-8b2b-4b04-84ed-99e96902b9fd/volumes/kubernetes.io~projected/kube-api-access-98nsz DeviceMajor:0 DeviceMinor:408 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c80ccf99-2313-400b-8477-2f14a8d08c50/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5/volumes/kubernetes.io~projected/kube-api-access-x497z DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:321 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:15e355a67b6192a MacAddress:42:33:f1:f2:8d:03 Speed:10000 Mtu:8901} {Name:350fe15d4147c9e MacAddress:b6:47:e0:6e:fd:7a Speed:10000 Mtu:8901} {Name:38908d5cf1a64d7 MacAddress:1e:b4:3c:d8:3d:9e Speed:10000 Mtu:8901} {Name:41a737dbb888e66 MacAddress:86:54:79:ba:1e:0a Speed:10000 Mtu:8901} {Name:4958e8f74e84f4f MacAddress:7e:aa:8d:5e:71:63 Speed:10000 Mtu:8901} {Name:4bd18deee211dd6 MacAddress:d6:44:c5:5a:f1:67 Speed:10000 Mtu:8901} {Name:4c30785328c0669 MacAddress:c2:0a:f7:54:4e:85 Speed:10000 Mtu:8901} {Name:54197d9403fcd65 MacAddress:da:1e:55:18:cc:2d Speed:10000 Mtu:8901} {Name:65616deff14cbaa MacAddress:de:49:b4:b1:65:e3 Speed:10000 Mtu:8901} {Name:828e9407a613b2f MacAddress:9e:8d:37:4b:a3:7e Speed:10000 Mtu:8901} {Name:944ee0c33a721dc MacAddress:32:9e:2f:a6:0b:fe Speed:10000 Mtu:8901} {Name:ae4e38f6f653ae4 MacAddress:4e:13:8c:a2:cb:33 Speed:10000 Mtu:8901} {Name:b827d3d30e2230a MacAddress:7e:c7:29:d7:7b:84 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:cf:27:7f:4f Speed:0 Mtu:9001} {Name:br-int MacAddress:7e:50:08:ba:fc:e7 Speed:0 Mtu:8901} {Name:cc0f7ed2aba9170 MacAddress:b2:b2:fe:c8:5f:c3 Speed:10000 Mtu:8901} {Name:e11ef808e59e662 MacAddress:aa:7c:2a:81:93:bb Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:cf:27:7f:4f Speed:0 Mtu:9001} {Name:fef341aef37e1c3 MacAddress:02:57:ed:dd:e4:ac Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:c6:61:08:6a:11:01 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:f6:52:d0:b4:6b:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 03:29:10.247234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.246878    5906 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 03:29:10.247234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.246958    5906 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 03:29:10.247357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247327    5906 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 03:29:10.247741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247360    5906 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-108.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 03:29:10.247741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247710    5906 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 03:29:10.247741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247724    5906 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 03:29:10.247741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247741    5906 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:29:10.247958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247770    5906 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:29:10.247958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.247898    5906 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 03:29:10.248201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248109    5906 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 03:29:10.248358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248327    5906 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 03:29:10.248408 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248384    5906 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 03:29:10.248408 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248405    5906 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 03:29:10.248513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248422    5906 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 03:29:10.248513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248436    5906 kubelet.go:405] "Adding apiserver pod source"
Oct 31 03:29:10.248513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.248449    5906 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 03:29:10.249654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.249582    5906 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 03:29:10.249654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.249599    5906 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:29:10.250590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.250535    5906 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 03:29:10.274402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.274371    5906 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 03:29:10.274402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.274394    5906 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:29:10.274660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.274562    5906 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 03:29:10.274864 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.274844    5906 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 03:29:10.275069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.274887    5906 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 03:29:10.275069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275044    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 03:29:10.275069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275063    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275074    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275083    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275093    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275102    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275112    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275122    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275134    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275145    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275158    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275172    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275224    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 03:29:10.275237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.275234    5906 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 03:29:10.279618 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.279598    5906 server.go:1263] "Started kubelet"
Oct 31 03:29:10.281105 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.279995    5906 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 03:29:10.281105 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.280066    5906 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 03:29:10.281105 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.280111    5906 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 03:29:10.281105 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.280657    5906 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 03:29:10.280085 ip-10-0-129-108 systemd[1]: Started Kubernetes Kubelet.
Oct 31 03:29:10.283827 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.283811    5906 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 03:29:10.284336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.284319    5906 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 03:29:10.284521 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.284496    5906 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:29:10.284687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.284544    5906 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 03:29:10.284687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.284552    5906 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 03:29:10.284878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.284776    5906 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 03:29:10.284878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.284543    5906 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 03:29:10.285149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.285103    5906 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 03:22:59 +0000 UTC" deadline="2028-05-26 18:14:17.324679961 +0000 UTC"
Oct 31 03:29:10.285149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.285134    5906 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13766h45m7.039548122s"
Oct 31 03:29:10.287051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.286809    5906 factory.go:55] Registering systemd factory
Oct 31 03:29:10.287051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.286826    5906 factory.go:223] Registration of the systemd container factory successfully
Oct 31 03:29:10.290227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.290177    5906 factory.go:153] Registering CRI-O factory
Oct 31 03:29:10.290227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.290221    5906 factory.go:223] Registration of the crio container factory successfully
Oct 31 03:29:10.290386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.290279    5906 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 03:29:10.290386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.290299    5906 factory.go:103] Registering Raw factory
Oct 31 03:29:10.290386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.290315    5906 manager.go:1199] Started watching for new ooms in manager
Oct 31 03:29:10.292014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.290736    5906 manager.go:319] Starting recovery of all containers
Oct 31 03:29:10.302723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302652    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0acad573-0b62-497a-9068-546a37bbc5ea" volumeName="kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 03:29:10.302723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302726    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302746    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302765    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9987cbad-f4ba-43be-91f7-f840a6fe0411" volumeName="kubernetes.io/configmap/9987cbad-f4ba-43be-91f7-f840a6fe0411-iptables-alerter-script" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302786    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35631c59-2c9c-4672-af14-6f7f6734fc24" volumeName="kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302805    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/empty-dir/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-tmp" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302823    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/projected/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-kube-api-access-rb58p" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302864    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302887    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c80ccf99-2313-400b-8477-2f14a8d08c50" volumeName="kubernetes.io/projected/c80ccf99-2313-400b-8477-2f14a8d08c50-kube-api-access-269hl" seLinuxMountContext=""
Oct 31 03:29:10.302916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302905    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35631c59-2c9c-4672-af14-6f7f6734fc24" volumeName="kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302922    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302941    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0ca45c9-f001-466d-82b0-852e1e486661" volumeName="kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302955    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302970    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c04cb90b-5446-4325-a368-a54295d49f33" volumeName="kubernetes.io/secret/c04cb90b-5446-4325-a368-a54295d49f33-agent-certs" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.302984    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42f88ad0-2936-4d56-9fef-a74e38d0c770" volumeName="kubernetes.io/projected/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-api-access-q6hs9" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303002    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="762debf9-cb16-4dc7-9923-c345e031f290" volumeName="kubernetes.io/configmap/762debf9-cb16-4dc7-9923-c345e031f290-serviceca" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303030    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87c006a7-67b8-4a1d-af88-d6a66fe6e2b5" volumeName="kubernetes.io/projected/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-kube-api-access-x497z" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303044    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303055    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303077    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303089    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/empty-dir/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-audit-log" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303100    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a7862a2-9e10-46d0-b7e9-929eb6158d3d" volumeName="kubernetes.io/projected/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kube-api-access-7gnm2" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303113    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303137    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 31 03:29:10.303149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303148    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" volumeName="kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303163    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303175    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303187    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42f88ad0-2936-4d56-9fef-a74e38d0c770" volumeName="kubernetes.io/secret/42f88ad0-2936-4d56-9fef-a74e38d0c770-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303221    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-tls" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303234    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa2ff59e-e274-44b6-968e-49cc6a16f1f3" volumeName="kubernetes.io/projected/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-kube-api-access-77fcb" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303245    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303273    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0acad573-0b62-497a-9068-546a37bbc5ea" volumeName="kubernetes.io/projected/0acad573-0b62-497a-9068-546a37bbc5ea-kube-api-access-lmvtk" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303289    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303302    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303326    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0ca45c9-f001-466d-82b0-852e1e486661" volumeName="kubernetes.io/projected/d0ca45c9-f001-466d-82b0-852e1e486661-kube-api-access-nfmh5" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303340    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/projected/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-kube-api-access-tlgqk" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303353    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303367    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24753e34-7be9-4a7e-8629-622b22fde606" volumeName="kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303378    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa2ff59e-e274-44b6-968e-49cc6a16f1f3" volumeName="kubernetes.io/secret/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-cert" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303393    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3874f6-24f2-4b6b-a99f-ee7c356dd01e" volumeName="kubernetes.io/secret/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-metrics-tls" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303407    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35631c59-2c9c-4672-af14-6f7f6734fc24" volumeName="kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303419    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303431    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303442    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303453    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303466    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35631c59-2c9c-4672-af14-6f7f6734fc24" volumeName="kubernetes.io/empty-dir/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-textfile" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303478    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42f88ad0-2936-4d56-9fef-a74e38d0c770" volumeName="kubernetes.io/empty-dir/42f88ad0-2936-4d56-9fef-a74e38d0c770-data-volume" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303491    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303503    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303515    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c04cb90b-5446-4325-a368-a54295d49f33" volumeName="kubernetes.io/configmap/c04cb90b-5446-4325-a368-a54295d49f33-konnectivity-ca" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303531    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2467cba4-f180-4a23-bb6a-5a67c8e86df4" volumeName="kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-daemon-config" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303543    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303559    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cd7aa60-8222-467f-88e2-88cb44ff12a5" volumeName="kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-tls" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303573    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303586    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="981be9db-6718-42fc-9076-8262bc218c66" volumeName="kubernetes.io/empty-dir/981be9db-6718-42fc-9076-8262bc218c66-tmp-dir" seLinuxMountContext=""
Oct 31 03:29:10.303603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303599    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303641    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303654    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35631c59-2c9c-4672-af14-6f7f6734fc24" volumeName="kubernetes.io/projected/35631c59-2c9c-4672-af14-6f7f6734fc24-kube-api-access-msp4w" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303666    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303682    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303695    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303708    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303720    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c80ccf99-2313-400b-8477-2f14a8d08c50" volumeName="kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-tmp" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303736    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="111ff747-5e37-43eb-8488-ea2ebe3432cc" volumeName="kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303749    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cd7aa60-8222-467f-88e2-88cb44ff12a5" volumeName="kubernetes.io/configmap/5cd7aa60-8222-467f-88e2-88cb44ff12a5-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303763    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3874f6-24f2-4b6b-a99f-ee7c356dd01e" volumeName="kubernetes.io/projected/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-kube-api-access-l95pf" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303775    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303787    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303800    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303812    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303824    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303837    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303853    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87c006a7-67b8-4a1d-af88-d6a66fe6e2b5" volumeName="kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-env-overrides" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303867    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303880    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="111ff747-5e37-43eb-8488-ea2ebe3432cc" volumeName="kubernetes.io/projected/111ff747-5e37-43eb-8488-ea2ebe3432cc-kube-api-access-wpm6t" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303891    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3874f6-24f2-4b6b-a99f-ee7c356dd01e" volumeName="kubernetes.io/configmap/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-config-volume" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303903    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303916    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303931    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303943    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303956    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cd7aa60-8222-467f-88e2-88cb44ff12a5" volumeName="kubernetes.io/projected/5cd7aa60-8222-467f-88e2-88cb44ff12a5-kube-api-access-4ls5m" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303969    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-grpc-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303982    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87c006a7-67b8-4a1d-af88-d6a66fe6e2b5" volumeName="kubernetes.io/secret/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.303995    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0ca45c9-f001-466d-82b0-852e1e486661" volumeName="kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304007    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0ca45c9-f001-466d-82b0-852e1e486661" volumeName="kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304020    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="35631c59-2c9c-4672-af14-6f7f6734fc24" volumeName="kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304032    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="762debf9-cb16-4dc7-9923-c345e031f290" volumeName="kubernetes.io/projected/762debf9-cb16-4dc7-9923-c345e031f290-kube-api-access-wbdbd" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304045    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304059    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304080    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66b0fafb-8b2b-4b04-84ed-99e96902b9fd" volumeName="kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304092    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304107    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87c006a7-67b8-4a1d-af88-d6a66fe6e2b5" volumeName="kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304121    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304134    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304149    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c80ccf99-2313-400b-8477-2f14a8d08c50" volumeName="kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-tuned" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304164    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" volumeName="kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304176    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304188    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304221    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304235    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304246    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304261    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9987cbad-f4ba-43be-91f7-f840a6fe0411" volumeName="kubernetes.io/projected/9987cbad-f4ba-43be-91f7-f840a6fe0411-kube-api-access-5kpwr" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304275    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304290    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0acad573-0b62-497a-9068-546a37bbc5ea" volumeName="kubernetes.io/configmap/0acad573-0b62-497a-9068-546a37bbc5ea-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304303    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304315    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304329    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a647986f-3521-49de-950c-fb705df308ab" volumeName="kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304345    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0ca45c9-f001-466d-82b0-852e1e486661" volumeName="kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304359    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" volumeName="kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304375    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2467cba4-f180-4a23-bb6a-5a67c8e86df4" volumeName="kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cni-binary-copy" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304389    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304401    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="87c006a7-67b8-4a1d-af88-d6a66fe6e2b5" volumeName="kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304464    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304476    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d0ca45c9-f001-466d-82b0-852e1e486661" volumeName="kubernetes.io/empty-dir/d0ca45c9-f001-466d-82b0-852e1e486661-volume-directive-shadow" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304484    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0acad573-0b62-497a-9068-546a37bbc5ea" volumeName="kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304493    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42f88ad0-2936-4d56-9fef-a74e38d0c770" volumeName="kubernetes.io/configmap/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304503    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bace1d01-5c41-4521-91b4-7ce780044fbe" volumeName="kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304511    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 31 03:29:10.304471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304519    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dc6feeb-2db4-4605-b2d8-6220c8d27c9c" volumeName="kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-client-ca-bundle" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304531    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cd7aa60-8222-467f-88e2-88cb44ff12a5" volumeName="kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304539    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/configmap/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-metrics-client-ca" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304548    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304556    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3874f6-24f2-4b6b-a99f-ee7c356dd01e" volumeName="kubernetes.io/empty-dir/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-tmp-dir" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304566    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2467cba4-f180-4a23-bb6a-5a67c8e86df4" volumeName="kubernetes.io/projected/2467cba4-f180-4a23-bb6a-5a67c8e86df4-kube-api-access-sthkd" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304573    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6862e1ee-e114-483c-b37a-1ce7ddeb52aa" volumeName="kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304582    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="981be9db-6718-42fc-9076-8262bc218c66" volumeName="kubernetes.io/projected/981be9db-6718-42fc-9076-8262bc218c66-kube-api-access-wvntl" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304590    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3dbbc148-54de-4755-ba0b-8dee372b9917" volumeName="kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304600    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91095503-a34e-402b-9017-84b2a5379894" volumeName="kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304611    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" volumeName="kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-binary-copy" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304620    5906 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" volumeName="kubernetes.io/projected/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-kube-api-access-xgq7n" seLinuxMountContext=""
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304629    5906 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 03:29:10.307038 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.304636    5906 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 03:29:10.341817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.341777    5906 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 03:29:10.343749 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.343721    5906 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 03:29:10.343921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.343907    5906 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 03:29:10.344043 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.344032    5906 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 03:29:10.344175 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.344153    5906 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 03:29:10.384869 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.384839    5906 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-129-108.us-west-2.compute.internal\" not found"
Oct 31 03:29:10.408732 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.408684    5906 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="010030d5385bf68e31959e271bde80917b00291b7ca6c1bf70b3f447507e402d" exitCode=0
Oct 31 03:29:10.408732 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.408726    5906 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="9ad34fb6e06518693f7d2e5d963cce24b36e235bef371aae4556647d3e381aa1" exitCode=0
Oct 31 03:29:10.408732 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.408739    5906 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="577dc403f1a521e46c7e1b0168fc287bc8696a20d00c451c2ada03b7727198f0" exitCode=0
Oct 31 03:29:10.409014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.408752    5906 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="e62d89391360705331a9940f4fe59d7c6bc3c51b0605b70c1ae2c07dbc72fc17" exitCode=0
Oct 31 03:29:10.409014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.408762    5906 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="e0638635444f0a376a20affeb6efb9d0e8f4c3140017f33c2710ee67482dcb8a" exitCode=0
Oct 31 03:29:10.409014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.408775    5906 generic.go:356] "Generic (PLEG): container finished" podID="fd31cb71-d1cb-4377-b6ef-2c9e08b1c887" containerID="bea426325ad284005770e14c14fcf4f720ef64809f7458fa3e0bf51d8f1ace70" exitCode=0
Oct 31 03:29:10.413520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.413483    5906 generic.go:356] "Generic (PLEG): container finished" podID="f00518c13840ee186767099c11c28d53" containerID="2edee0761bcab0ae5348f9e21563021f4888a7d6273ec165031993fa8f22fd0b" exitCode=0
Oct 31 03:29:10.444323 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.444290    5906 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 31 03:29:10.448972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.448911    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea" exitCode=0
Oct 31 03:29:10.460970 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.460875    5906 manager.go:324] Recovery completed
Oct 31 03:29:10.469555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.469519    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88" exitCode=0
Oct 31 03:29:10.474695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.474658    5906 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 03:29:10.477427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.477396    5906 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:29:10.478496 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.478469    5906 generic.go:356] "Generic (PLEG): container finished" podID="35631c59-2c9c-4672-af14-6f7f6734fc24" containerID="292718d56dd78d2a145245d931d3f0b422c8ec50aa7b5b46d54949e0e1c8c3c6" exitCode=0
Oct 31 03:29:10.495375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495340    5906 cpu_manager.go:226] "Starting" policy="none"
Oct 31 03:29:10.495375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495361    5906 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 03:29:10.495569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495389    5906 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 03:29:10.495569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495544    5906 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 03:29:10.495631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495559    5906 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 03:29:10.495631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495584    5906 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 03:29:10.495631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495592    5906 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 03:29:10.495631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495598    5906 policy_none.go:50] "Start"
Oct 31 03:29:10.495631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495608    5906 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 03:29:10.495631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495618    5906 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 03:29:10.495843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495768    5906 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 03:29:10.495843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495776    5906 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 03:29:10.495843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.495781    5906 policy_none.go:44] "Start"
Oct 31 03:29:10.501747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.501710    5906 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 03:29:10.506151 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.506130    5906 manager.go:341] "Starting Device Plugin manager"
Oct 31 03:29:10.506914 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.506165    5906 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 03:29:10.506914 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.506917    5906 server.go:87] "Starting device plugin registration server"
Oct 31 03:29:10.507287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507273    5906 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 03:29:10.507398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507286    5906 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 03:29:10.507398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507376    5906 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 03:29:10.507524 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507482    5906 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 03:29:10.507584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507540    5906 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 03:29:10.507584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507550    5906 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 03:29:10.507682 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.507607    5906 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T03:29:10.507522447Z","UUID":"cace286b-1a70-4efc-b707-fdb9247295c3","Handler":null,"Name":"","Endpoint":""}
Oct 31 03:29:10.512514 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.512488    5906 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 03:29:10.512637 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.512535    5906 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 03:29:10.513778 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.513757    5906 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 03:29:10.537269 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.534498    5906 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:29:10.569883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.569853    5906 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:29:10.577656 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.577618    5906 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:29:10.607941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.607903    5906 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:29:10.610175 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.610147    5906 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:29:10.610313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.610182    5906 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:29:10.610313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.610192    5906 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-129-108.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:29:10.610313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.610263    5906 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.644514 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.644416    5906 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"]
Oct 31 03:29:10.645246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645199    5906 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.645380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645190    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"f00518c13840ee186767099c11c28d53","Type":"ContainerStarted","Data":"e38706dabbe0e2e6f8d08fcfc28ccd783837cff073c760b3e44a5b7fbe865010"}
Oct 31 03:29:10.645380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645334    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"f00518c13840ee186767099c11c28d53","Type":"ContainerDied","Data":"2edee0761bcab0ae5348f9e21563021f4888a7d6273ec165031993fa8f22fd0b"}
Oct 31 03:29:10.645380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645360    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"f00518c13840ee186767099c11c28d53","Type":"ContainerStarted","Data":"27e2b2efb065a28251ec236f0ca2c5a40a8ee85aa5320608f0a1ab573116aa59"}
Oct 31 03:29:10.645380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645376    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"79d038c1345182a8a45cdaeaa396f9d8","Type":"ContainerStarted","Data":"71e163ea2d8bd26853bae5559fcb77428423dcf3444b5e99232cb4abf391a558"}
Oct 31 03:29:10.645583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645394    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal" event={"ID":"79d038c1345182a8a45cdaeaa396f9d8","Type":"ContainerStarted","Data":"ac827466574e80904c0d0bd7a1d6755fcb5fb45b26d0544850187549fc5f511c"}
Oct 31 03:29:10.645583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.645410    5906 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.710406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.710378    5906 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:29:10.710540 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.710434    5906 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.710941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.710923    5906 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:29:10.711000 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:10.710958    5906 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.723564 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.723542    5906 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.723635 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.723600    5906 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.776729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.776687    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.776729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.776729    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.776945 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.776759    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79d038c1345182a8a45cdaeaa396f9d8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"79d038c1345182a8a45cdaeaa396f9d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.877792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.877749    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.877792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.877788    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.877792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.877806    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79d038c1345182a8a45cdaeaa396f9d8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"79d038c1345182a8a45cdaeaa396f9d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.878031 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.877853    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79d038c1345182a8a45cdaeaa396f9d8-config\") pod \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"79d038c1345182a8a45cdaeaa396f9d8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.878031 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.877861    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:10.878031 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:10.877859    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f00518c13840ee186767099c11c28d53-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" (UID: \"f00518c13840ee186767099c11c28d53\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:11.249938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.249901    5906 apiserver.go:52] "Watching apiserver"
Oct 31 03:29:11.408917 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.408877    5906 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 03:29:11.409693 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.409639    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-mvgh4","openshift-image-registry/node-ca-6m89w","openshift-ingress-canary/ingress-canary-94ztd","openshift-dns/node-resolver-stklk","openshift-insights/insights-runtime-extractor-gzcxv","openshift-monitoring/alertmanager-main-0","openshift-console/console-59c57d5c75-2rvq9","openshift-monitoring/thanos-querier-8444c575f6-l95rj","openshift-monitoring/node-exporter-q5pft","openshift-network-diagnostics/network-check-target-7qbtw","openshift-network-operator/iptables-alerter-zv4qg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx","openshift-console/console-dc44455cf-tknzm","openshift-ovn-kubernetes/ovnkube-node-rxfjs","openshift-monitoring/prometheus-operator-9cd89b74c-shcbg","openshift-console/console-6db55f7dc6-jpb5c","openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal","openshift-multus/multus-zfvc8","kube-system/global-pull-secret-syncer-xgj98","openshift-dns/dns-default-8sd9m","openshift-multus/network-metrics-daemon-cwdx2","kube-system/konnectivity-agent-p8r6z","openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h","kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-kggl4","openshift-monitoring/metrics-server-79fcb4cc8-rjl9f","openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:29:11.433093 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.433054    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.465900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.465863    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 03:29:11.465900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.465866    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lj989\""
Oct 31 03:29:11.465900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.465873    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 03:29:11.466125 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.465880    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hp5ds\""
Oct 31 03:29:11.466125 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.465920    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-8f2x4\""
Oct 31 03:29:11.466187 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466123    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hztlx\""
Oct 31 03:29:11.466241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466224    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zvcgd\""
Oct 31 03:29:11.466309 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466297    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-dlcmg\""
Oct 31 03:29:11.466842 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466824    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.466927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466831    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 03:29:11.466927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466891    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.466927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466894    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.467072 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.466939    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.467072 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467045    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.467162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467068    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 03:29:11.467162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467104    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.467162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467109    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.467162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467110    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.467162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467134    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 03:29:11.467162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467117    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 03:29:11.467427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467144    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.467427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467335    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.467491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.467468    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 03:29:11.482813 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.482791    5906 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:11.482927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.482827    5906 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:11.487705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.487689    5906 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 03:29:11.544384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.544315    5906 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:29:11.544384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.544324    5906 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:29:11.544384 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:11.544380    5906 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:11.544584 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:11.544394    5906 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-108.us-west-2.compute.internal"
Oct 31 03:29:11.571586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571557    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.571790 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571556    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571619    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571623    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571712    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571728    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571745    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571780    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571856    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571860    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 03:29:11.571894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571878    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.572342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571962    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 03:29:11.572342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.571988    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.572342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572050    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.572476 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572360    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 31 03:29:11.572476 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572387    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 31 03:29:11.572566 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572526    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 31 03:29:11.572653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572634    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 31 03:29:11.572759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572744    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 03:29:11.572836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572824    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 31 03:29:11.572919 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572906    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 03:29:11.572948 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572940    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 03:29:11.573001 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.572988    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 31 03:29:11.573094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573080    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 03:29:11.573151 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573087    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.573151 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573121    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 03:29:11.573151 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573145    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 03:29:11.573307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573226    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 03:29:11.573307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573262    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9sxx\""
Oct 31 03:29:11.573307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573269    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 03:29:11.573416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573327    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 03:29:11.573416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573340    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 03:29:11.573501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573473    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.573549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573499    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:29:11.573549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573505    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 03:29:11.573549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573531    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 31 03:29:11.573549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573545    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573581    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573646    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573680    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573695    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573697    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573711    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573726    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573746    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573755    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573768    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573756    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 31 03:29:11.573778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573762    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573801    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573830    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573851    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573866    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ktggl\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573875    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4m9nhhr1ar1k5\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573855    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z5hqt\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573901    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573926    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8641tjbr9hbo8\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573938    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573944    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573954    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573981    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwkt\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573997    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574068    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574135    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hp8t5\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574154    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdz67\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574155    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-dvnpq\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574156    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574196    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574237    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2rc66\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574247    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fs258\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574272    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574290    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 03:29:11.574289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574304    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-npq57\""
Oct 31 03:29:11.575080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574335    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-24j69\""
Oct 31 03:29:11.575080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574369    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 03:29:11.575080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.573940    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 03:29:11.575080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574454    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 31 03:29:11.575080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.574469    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 31 03:29:11.581870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.581848    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.581966 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.581879    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-wtmp\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.581966 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.581897    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x497z\" (UniqueName: \"kubernetes.io/projected/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-kube-api-access-x497z\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.582094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582079    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ca45c9-f001-466d-82b0-852e1e486661-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.582146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582104    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cnibin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.582146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582125    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-hostroot\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.582276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582152    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.582276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582229    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ca45c9-f001-466d-82b0-852e1e486661-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.582276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582243    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.582276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582257    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582310    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-env-overrides\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582331    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582355    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582377    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-host\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582393    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cnibin\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582402    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582409    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.582461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582436    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582520    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-env-overrides\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582564    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582600    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-binary-copy\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582613    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582670    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-config\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582695    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582712    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582731    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582758    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582783    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-metrics-client-ca\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.582831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582826    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582856    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981be9db-6718-42fc-9076-8262bc218c66-tmp-dir\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582884    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582901    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b9rvf\""
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582908    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582932    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582961    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-fn57l\""
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.582962    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-tmp\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583048    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583090    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-audit-log\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583113    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njklp\""
Oct 31 03:29:11.583468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583146    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-audit-log\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.583973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583548    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-tmp\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.583973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583585    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 03:29:11.583973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583697    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.583973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583787    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-metrics-client-ca\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.583973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583953    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.584229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.583121    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cni-binary-copy\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.584229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584081    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-config\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.584229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.584229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584128    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.584229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584160    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-269hl\" (UniqueName: \"kubernetes.io/projected/c80ccf99-2313-400b-8477-2f14a8d08c50-kube-api-access-269hl\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.584229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584189    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-systemd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584231    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-etc-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584263    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584287    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584324    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584336    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cni-binary-copy\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584352    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-tuned\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584352    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584409    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584450    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c04cb90b-5446-4325-a368-a54295d49f33-agent-certs\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584464    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.584520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584481    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584522    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584690    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584716    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584728    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6hs9\" (UniqueName: \"kubernetes.io/projected/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-api-access-q6hs9\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584732    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584915    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-tuned\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.585029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.584915    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/981be9db-6718-42fc-9076-8262bc218c66-tmp-dir\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.585356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585190    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 31 03:29:11.585356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585305    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.585461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585385    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dl6f5\""
Oct 31 03:29:11.585678 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585631    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-tls\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.585678 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585669    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.585787 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585676    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-systemd-units\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.585787 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585718    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd7aa60-8222-467f-88e2-88cb44ff12a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.585893 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585840    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/981be9db-6718-42fc-9076-8262bc218c66-hosts-file\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.585956    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586047    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5cd7aa60-8222-467f-88e2-88cb44ff12a5-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586180    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762debf9-cb16-4dc7-9923-c345e031f290-serviceca\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586357    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgq7n\" (UniqueName: \"kubernetes.io/projected/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-kube-api-access-xgq7n\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586365    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-tls\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586402    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-ovn\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.586473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586438    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.586865 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586676    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.586865 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586736    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.586865 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586792    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmvtk\" (UniqueName: \"kubernetes.io/projected/0acad573-0b62-497a-9068-546a37bbc5ea-kube-api-access-lmvtk\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.587016 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586946    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-multus\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.587016 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.586991    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.587112 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587049    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/762debf9-cb16-4dc7-9923-c345e031f290-serviceca\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.587410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587257    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.587410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587297    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9987cbad-f4ba-43be-91f7-f840a6fe0411-host-slash\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.587410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587323    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpwr\" (UniqueName: \"kubernetes.io/projected/9987cbad-f4ba-43be-91f7-f840a6fe0411-kube-api-access-5kpwr\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.587410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587354    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.587410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587385    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587417    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587446    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587471    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-tmp\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587503    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587533    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sthkd\" (UniqueName: \"kubernetes.io/projected/2467cba4-f180-4a23-bb6a-5a67c8e86df4-kube-api-access-sthkd\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587562    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587575    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587586    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587613    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762debf9-cb16-4dc7-9923-c345e031f290-host\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587641    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-root\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.587676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587670    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587701    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587727    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587755    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587784    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysconfig\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.587783    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588009    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588026    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.588200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588110    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c80ccf99-2313-400b-8477-2f14a8d08c50-tmp\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588236    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588387    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovn-node-metrics-cert\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588454    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-kubernetes\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588489    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-config-volume\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588521    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-metrics-tls\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588551    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.588587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588581    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42f88ad0-2936-4d56-9fef-a74e38d0c770-crio-socket\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.588903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588614    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.588903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588848    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.588903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588868    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-metrics-tls\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.589047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.588945    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.589047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589013    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-config-volume\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589158    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589187    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589306    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-socket-dir-parent\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589322    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589341    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-netns\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589371    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-conf-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589427    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589463    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-registration-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589477    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589514    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589593    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589640    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-tls\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.589728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589698    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msp4w\" (UniqueName: \"kubernetes.io/projected/35631c59-2c9c-4672-af14-6f7f6734fc24-kube-api-access-msp4w\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589896    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-tls\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589957    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.589989    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-conf\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590055    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-metrics-client-ca\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590088    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590119    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-system-cni-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590149    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590314    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590338    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590361    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-systemd\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.590458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590419    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590516    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fcb\" (UniqueName: \"kubernetes.io/projected/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-kube-api-access-77fcb\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590557    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590591    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590645    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-metrics-client-ca\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590680    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590789    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/24753e34-7be9-4a7e-8629-622b22fde606-original-pull-secret\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590855    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590892    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.590961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.590960    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-daemon-config\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.591411 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591366    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.591411 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591374    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f88ad0-2936-4d56-9fef-a74e38d0c770-data-volume\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.591506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591418    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-netd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.591506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591464    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/42f88ad0-2936-4d56-9fef-a74e38d0c770-data-volume\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.591506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591481    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-daemon-config\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.591506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591486    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.591694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591518    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpm6t\" (UniqueName: \"kubernetes.io/projected/111ff747-5e37-43eb-8488-ea2ebe3432cc-kube-api-access-wpm6t\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:11.591694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591536    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-client-certs\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.591694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591555    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-netns\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.591694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591649    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-log-socket\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.591694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591672    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gnm2\" (UniqueName: \"kubernetes.io/projected/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kube-api-access-7gnm2\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.591694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591689    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591713    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591727    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591758    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-secret-metrics-server-client-certs\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591829    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-sys\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591859    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591896    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-kubelet\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591912    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591930    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.591972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.591966    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592011    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-sys-fs\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592046    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592049    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592086    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592105    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592123    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592172    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592218    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-var-lib-kubelet\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592224    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592249    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-accelerators-collector-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592275    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-bin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592298    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-run\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592341    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592359    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592369    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592389    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592397    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-client-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592404    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592434    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-kubelet-config\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.592447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592440    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-accelerators-collector-config\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592478    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-script-lib\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592528    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592572    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-multus-certs\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592591    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592649    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbdbd\" (UniqueName: \"kubernetes.io/projected/762debf9-cb16-4dc7-9923-c345e031f290-kube-api-access-wbdbd\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592681    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-tmp-dir\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592703    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-ovnkube-script-lib\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592706    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592758    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-var-lib-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592788    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592783    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0acad573-0b62-497a-9068-546a37bbc5ea-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592815    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-sys\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592843    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvntl\" (UniqueName: \"kubernetes.io/projected/981be9db-6718-42fc-9076-8262bc218c66-kube-api-access-wvntl\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592870    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-tmp-dir\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592878    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-system-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592801    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592924    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-k8s-cni-cncf-io\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592947    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592951    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592963    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-modprobe-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.592969    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593072    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95pf\" (UniqueName: \"kubernetes.io/projected/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-kube-api-access-l95pf\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593082    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593108    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-textfile\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593134    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-grpc-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593152    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-etc-kubernetes\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593181    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42f88ad0-2936-4d56-9fef-a74e38d0c770-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593256    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-metrics-server-audit-profiles\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593256    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-textfile\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593287    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-node-log\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593314    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593340    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593365    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-grpc-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593389    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ls5m\" (UniqueName: \"kubernetes.io/projected/5cd7aa60-8222-467f-88e2-88cb44ff12a5-kube-api-access-4ls5m\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593423    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.593443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593427    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/42f88ad0-2936-4d56-9fef-a74e38d0c770-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593464    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593501    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-metrics-server-audit-profiles\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593534    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-cert\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593561    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5cd7aa60-8222-467f-88e2-88cb44ff12a5-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593600    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593633    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593634    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlgqk\" (UniqueName: \"kubernetes.io/projected/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-kube-api-access-tlgqk\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593669    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-kubelet\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593699    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593729    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-slash\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593760    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593785    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-cert\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593786    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acad573-0b62-497a-9068-546a37bbc5ea-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593820    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-socket-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593850    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-device-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593899    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593948    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593959    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0acad573-0b62-497a-9068-546a37bbc5ea-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.593987    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594032    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594067    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb58p\" (UniqueName: \"kubernetes.io/projected/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-kube-api-access-rb58p\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594095    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-bin\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594140    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594197    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594263    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594279    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594381    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-lib-modules\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594415    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594424    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594459    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594463    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfmh5\" (UniqueName: \"kubernetes.io/projected/d0ca45c9-f001-466d-82b0-852e1e486661-kube-api-access-nfmh5\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594492    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594529    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594639    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594651    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594709    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594743    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c04cb90b-5446-4325-a368-a54295d49f33-konnectivity-ca\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594835    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594854    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594860    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/111ff747-5e37-43eb-8488-ea2ebe3432cc-metrics-certs\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594871    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.594863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594888    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594945    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.594953    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c04cb90b-5446-4325-a368-a54295d49f33-konnectivity-ca\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595007    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595040    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595048    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595040    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9987cbad-f4ba-43be-91f7-f840a6fe0411-iptables-alerter-script\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595093    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-dbus\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595124    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595136    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595151    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595227    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9987cbad-f4ba-43be-91f7-f840a6fe0411-iptables-alerter-script\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595366    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.595999    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 31 03:29:11.596178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.596024    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wtzlk\""
Oct 31 03:29:11.603503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.603484    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-secret-thanos-querier-tls\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.623355 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.623324    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 03:29:11.623506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.623491    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 31 03:29:11.623574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.623548    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8nlcv\""
Oct 31 03:29:11.623574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.623552    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 03:29:11.623639 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.623595    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7b9lc\""
Oct 31 03:29:11.623905 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.623879    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c8hm9198qkt62\""
Oct 31 03:29:11.624113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.624092    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.625402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.625379    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ca45c9-f001-466d-82b0-852e1e486661-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.625494 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.625476    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c04cb90b-5446-4325-a368-a54295d49f33-agent-certs\") pod \"konnectivity-agent-p8r6z\" (UID: \"c04cb90b-5446-4325-a368-a54295d49f33\") " pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:29:11.626341 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.626318    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 31 03:29:11.626635 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.626618    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 03:29:11.626689 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.626643    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 31 03:29:11.626689 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.626679    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 03:29:11.628344 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.628328    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.629506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.629487    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.629554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.629540    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.632661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.632641    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-client-ca-bundle\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.636373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.636349    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.639531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.639507    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.642552 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.642530    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x497z\" (UniqueName: \"kubernetes.io/projected/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-kube-api-access-x497z\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.642933 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.642917    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") pod \"alertmanager-main-0\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:29:11.643275 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.643257    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") pod \"console-59c57d5c75-2rvq9\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") " pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:11.644570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.644554    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6hs9\" (UniqueName: \"kubernetes.io/projected/42f88ad0-2936-4d56-9fef-a74e38d0c770-kube-api-access-q6hs9\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.645926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.645908    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-269hl\" (UniqueName: \"kubernetes.io/projected/c80ccf99-2313-400b-8477-2f14a8d08c50-kube-api-access-269hl\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.646637 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.646622    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgq7n\" (UniqueName: \"kubernetes.io/projected/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-kube-api-access-xgq7n\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.647044 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.647027    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmvtk\" (UniqueName: \"kubernetes.io/projected/0acad573-0b62-497a-9068-546a37bbc5ea-kube-api-access-lmvtk\") pod \"openshift-state-metrics-7bd7489957-t6q8h\" (UID: \"0acad573-0b62-497a-9068-546a37bbc5ea\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-t6q8h"
Oct 31 03:29:11.648235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.648201    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpwr\" (UniqueName: \"kubernetes.io/projected/9987cbad-f4ba-43be-91f7-f840a6fe0411-kube-api-access-5kpwr\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.648424 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.648408    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sthkd\" (UniqueName: \"kubernetes.io/projected/2467cba4-f180-4a23-bb6a-5a67c8e86df4-kube-api-access-sthkd\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.650284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.650268    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msp4w\" (UniqueName: \"kubernetes.io/projected/35631c59-2c9c-4672-af14-6f7f6734fc24-kube-api-access-msp4w\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.652047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.652020    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fcb\" (UniqueName: \"kubernetes.io/projected/aa2ff59e-e274-44b6-968e-49cc6a16f1f3-kube-api-access-77fcb\") pod \"ingress-canary-94ztd\" (UID: \"aa2ff59e-e274-44b6-968e-49cc6a16f1f3\") " pod="openshift-ingress-canary/ingress-canary-94ztd"
Oct 31 03:29:11.652175 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.652055    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpm6t\" (UniqueName: \"kubernetes.io/projected/111ff747-5e37-43eb-8488-ea2ebe3432cc-kube-api-access-wpm6t\") pod \"network-metrics-daemon-cwdx2\" (UID: \"111ff747-5e37-43eb-8488-ea2ebe3432cc\") " pod="openshift-multus/network-metrics-daemon-cwdx2"
Oct 31 03:29:11.652326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.652310    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gnm2\" (UniqueName: \"kubernetes.io/projected/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kube-api-access-7gnm2\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.652381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.652311    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") pod \"console-6db55f7dc6-jpb5c\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") " pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.652381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.652358    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98nsz\" (UniqueName: \"kubernetes.io/projected/66b0fafb-8b2b-4b04-84ed-99e96902b9fd-kube-api-access-98nsz\") pod \"network-check-target-7qbtw\" (UID: \"66b0fafb-8b2b-4b04-84ed-99e96902b9fd\") " pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:11.652591 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.652577    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") pod \"prometheus-k8s-0\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:11.654138 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.654121    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvntl\" (UniqueName: \"kubernetes.io/projected/981be9db-6718-42fc-9076-8262bc218c66-kube-api-access-wvntl\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.655675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.655654    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95pf\" (UniqueName: \"kubernetes.io/projected/1e3874f6-24f2-4b6b-a99f-ee7c356dd01e-kube-api-access-l95pf\") pod \"dns-default-8sd9m\" (UID: \"1e3874f6-24f2-4b6b-a99f-ee7c356dd01e\") " pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:11.655675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.655666    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbdbd\" (UniqueName: \"kubernetes.io/projected/762debf9-cb16-4dc7-9923-c345e031f290-kube-api-access-wbdbd\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.657384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.657354    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlgqk\" (UniqueName: \"kubernetes.io/projected/4dc6feeb-2db4-4605-b2d8-6220c8d27c9c-kube-api-access-tlgqk\") pod \"metrics-server-79fcb4cc8-rjl9f\" (UID: \"4dc6feeb-2db4-4605-b2d8-6220c8d27c9c\") " pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:11.657551 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.657532    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfmh5\" (UniqueName: \"kubernetes.io/projected/d0ca45c9-f001-466d-82b0-852e1e486661-kube-api-access-nfmh5\") pod \"kube-state-metrics-5dc8d7d89b-jsvx7\" (UID: \"d0ca45c9-f001-466d-82b0-852e1e486661\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-jsvx7"
Oct 31 03:29:11.657659 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.657642    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ls5m\" (UniqueName: \"kubernetes.io/projected/5cd7aa60-8222-467f-88e2-88cb44ff12a5-kube-api-access-4ls5m\") pod \"prometheus-operator-9cd89b74c-shcbg\" (UID: \"5cd7aa60-8222-467f-88e2-88cb44ff12a5\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-shcbg"
Oct 31 03:29:11.657705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.657681    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") pod \"console-dc44455cf-tknzm\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") " pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:11.657738 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.657681    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb58p\" (UniqueName: \"kubernetes.io/projected/6862e1ee-e114-483c-b37a-1ce7ddeb52aa-kube-api-access-rb58p\") pod \"thanos-querier-8444c575f6-l95rj\" (UID: \"6862e1ee-e114-483c-b37a-1ce7ddeb52aa\") " pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:11.676927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.676900    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:11.695788 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695761    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-systemd\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.695788 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695794    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-netd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695813    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-netns\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695827    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-log-socket\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695842    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-sys\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695882    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-sys\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695887    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-netd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695903    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-log-socket\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695903    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-netns\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695887    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-systemd\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695927    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-kubelet\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695957    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-kubelet\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.695978    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696009    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-sys-fs\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696028    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-etc-selinux\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696039    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-var-lib-kubelet\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696049    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-sys-fs\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696063    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-bin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696080    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-var-lib-kubelet\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696087    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-run\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696116    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696117    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-bin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696138    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-kubelet-config\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696158    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-run\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696160    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-multus-certs\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696168    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696183    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-var-lib-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696195    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-kubelet-config\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696223    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-sys\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696249    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-system-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696257    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-multus-certs\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696266    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-var-lib-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696273    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-k8s-cni-cncf-io\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696286    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-sys\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696297    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-modprobe-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696304    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-k8s-cni-cncf-io\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696315    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-system-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696327    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-etc-kubernetes\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696347    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-node-log\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696364    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.696367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696381    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-kubelet\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696385    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-etc-kubernetes\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696403    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-node-log\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696404    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-slash\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696426    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-kubelet\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696433    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-socket-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696432    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-modprobe-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696447    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-cni-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696438    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-slash\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696461    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-device-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696472    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-socket-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696482    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-bin\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696489    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-device-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696501    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696521    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-lib-modules\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696546    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-cni-bin\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696548    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696569    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696579    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696585    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-dbus\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696612    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-wtmp\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696626    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-lib-modules\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696630    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cnibin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696696    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/24753e34-7be9-4a7e-8629-622b22fde606-dbus\") pod \"global-pull-secret-syncer-xgj98\" (UID: \"24753e34-7be9-4a7e-8629-622b22fde606\") " pod="kube-system/global-pull-secret-syncer-xgj98"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696705    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-hostroot\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696702    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-cnibin\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696706    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-node-exporter-wtmp\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696734    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-hostroot\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696738    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696767    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-host\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696772    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696790    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cnibin\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696800    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-host\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696819    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696834    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-systemd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696853    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-etc-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696898    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-systemd-units\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696903    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-cnibin\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696909    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-systemd\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696918    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-etc-openvswitch\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696928    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-systemd-units\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696918    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/981be9db-6718-42fc-9076-8262bc218c66-hosts-file\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696945    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-d\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696959    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-ovn\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696976    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-multus\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696981    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/87c006a7-67b8-4a1d-af88-d6a66fe6e2b5-run-ovn\") pod \"ovnkube-node-rxfjs\" (UID: \"87c006a7-67b8-4a1d-af88-d6a66fe6e2b5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696957    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/981be9db-6718-42fc-9076-8262bc218c66-hosts-file\") pod \"node-resolver-stklk\" (UID: \"981be9db-6718-42fc-9076-8262bc218c66\") " pod="openshift-dns/node-resolver-stklk"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.696993    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9987cbad-f4ba-43be-91f7-f840a6fe0411-host-slash\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697006    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-var-lib-cni-multus\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697018    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762debf9-cb16-4dc7-9923-c345e031f290-host\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697028    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9987cbad-f4ba-43be-91f7-f840a6fe0411-host-slash\") pod \"iptables-alerter-zv4qg\" (UID: \"9987cbad-f4ba-43be-91f7-f840a6fe0411\") " pod="openshift-network-operator/iptables-alerter-zv4qg"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697039    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-root\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697053    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/762debf9-cb16-4dc7-9923-c345e031f290-host\") pod \"node-ca-6m89w\" (UID: \"762debf9-cb16-4dc7-9923-c345e031f290\") " pod="openshift-image-registry/node-ca-6m89w"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697058    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysconfig\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697070    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/35631c59-2c9c-4672-af14-6f7f6734fc24-root\") pod \"node-exporter-q5pft\" (UID: \"35631c59-2c9c-4672-af14-6f7f6734fc24\") " pod="openshift-monitoring/node-exporter-q5pft"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697078    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-kubernetes\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697084    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysconfig\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42f88ad0-2936-4d56-9fef-a74e38d0c770-crio-socket\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697115    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-socket-dir-parent\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697130    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-netns\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697147    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-conf-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697162    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697165    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/42f88ad0-2936-4d56-9fef-a74e38d0c770-crio-socket\") pod \"insights-runtime-extractor-gzcxv\" (UID: \"42f88ad0-2936-4d56-9fef-a74e38d0c770\") " pod="openshift-insights/insights-runtime-extractor-gzcxv"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697177    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-registration-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697193    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-kubernetes\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697198    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-socket-dir-parent\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697225    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-conf\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697232    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-multus-conf-dir\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697235    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697256    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2467cba4-f180-4a23-bb6a-5a67c8e86df4-host-run-netns\") pod \"multus-zfvc8\" (UID: \"2467cba4-f180-4a23-bb6a-5a67c8e86df4\") " pod="openshift-multus/multus-zfvc8"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697307    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-system-cni-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697307    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6a7862a2-9e10-46d0-b7e9-929eb6158d3d-registration-dir\") pod \"aws-ebs-csi-driver-node-qlwpx\" (UID: \"6a7862a2-9e10-46d0-b7e9-929eb6158d3d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qlwpx"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697335    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fd31cb71-d1cb-4377-b6ef-2c9e08b1c887-system-cni-dir\") pod \"multus-additional-cni-plugins-mvgh4\" (UID: \"fd31cb71-d1cb-4377-b6ef-2c9e08b1c887\") " pod="openshift-multus/multus-additional-cni-plugins-mvgh4"
Oct 31 03:29:11.697386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.697380    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c80ccf99-2313-400b-8477-2f14a8d08c50-etc-sysctl-conf\") pod \"tuned-kggl4\" (UID: \"c80ccf99-2313-400b-8477-2f14a8d08c50\") " pod="openshift-cluster-node-tuning-operator/tuned-kggl4"
Oct 31 03:29:11.733520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.733484    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:11.734444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.734424    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:29:11.814509 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:11.814434    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:12.234985 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:12.234900    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:12.370561 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:12.370525    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:12.487729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:12.487642    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db55f7dc6-jpb5c" event={"ID":"3dbbc148-54de-4755-ba0b-8dee372b9917","Type":"ContainerStarted","Data":"a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751"}
Oct 31 03:29:12.487729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:12.487700    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db55f7dc6-jpb5c" event={"ID":"3dbbc148-54de-4755-ba0b-8dee372b9917","Type":"ContainerStarted","Data":"4bd18deee211dd6dc6597de097aa941e0501ad95d9666191967976ef8e3511c5"}
Oct 31 03:29:12.581283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:12.581255    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:12.761406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:12.761380    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:13.042234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.040829    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:13.117951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.116931    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8sd9m"
Oct 31 03:29:13.123315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.123292    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:13.130241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.129440    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8444c575f6-l95rj"
Oct 31 03:29:13.376454 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.376367    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:13.495637 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.495599    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"}
Oct 31 03:29:13.495810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.495674    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"}
Oct 31 03:29:13.495810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.495703    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"}
Oct 31 03:29:13.495810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.495779    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"}
Oct 31 03:29:13.495810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.495802    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"}
Oct 31 03:29:13.496015 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.495826    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerStarted","Data":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"}
Oct 31 03:29:13.794512 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.794481    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:13.964171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:13.964133    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:15.155988 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:15.155950    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:29:15.753454 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:15.753420    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:15.758150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:15.758125    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:16.261815 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:16.261490    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:16.405499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:16.405465    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:16.610984 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:16.610916    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:16.611200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:16.611182    5906 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:29:17.566415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:17.566381    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:17.566884 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:17.566493    5906 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:29:17.570365 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:17.570343    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:29:17.744086 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:17.744045    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c57d5c75-2rvq9"]
Oct 31 03:29:19.382043 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:19.381961    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:19.523694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:19.523661    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:21.937835 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:21.937805    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p8r6z"
Oct 31 03:29:25.437555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:25.437521    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:25.971458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:25.971426    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:26.116301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:26.116268    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:30.334906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:30.334873    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rxfjs"
Oct 31 03:29:33.046436 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:33.046404    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:33.050410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:33.050378    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-79fcb4cc8-rjl9f"
Oct 31 03:29:35.575326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:35.575282    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:36.380555 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:36.380518    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:29:36.442164 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.442128    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-dc44455cf-tknzm" podUID="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" containerName="console" containerID="cri-o://47bb757ff7524da3958839b661189e02f3616fecb541e7212868624b8a8102c3" gracePeriod=15
Oct 31 03:29:36.582671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.582637    5906 generic.go:356] "Generic (PLEG): container finished" podID="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" containerID="47bb757ff7524da3958839b661189e02f3616fecb541e7212868624b8a8102c3" exitCode=0
Oct 31 03:29:36.583035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.582696    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc44455cf-tknzm" event={"ID":"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45","Type":"ContainerDied","Data":"47bb757ff7524da3958839b661189e02f3616fecb541e7212868624b8a8102c3"}
Oct 31 03:29:36.698533 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.698506    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:36.833950 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.833914    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834000    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834028    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834093    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834270 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834148    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834270 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834180    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834270 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834230    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") pod \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\" (UID: \"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45\") "
Oct 31 03:29:36.834494 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834470    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:29:36.834596 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834463    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:36.834690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834664    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:36.834889 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.834864    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:36.836685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.836661    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "kube-api-access-kdwf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:36.836776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.836682    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:36.836776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.836734    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config" pod "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" (UID: "f3bcb79d-3ca1-49f4-bbe3-19d10b620a45"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935308    5906 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935341    5906 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935351    5906 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-oauth-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935361    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdwf5\" (UniqueName: \"kubernetes.io/projected/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-kube-api-access-kdwf5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935372    5906 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-oauth-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935381    5906 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-console-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:36.935392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:36.935390    5906 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45-tmp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:37.587499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:37.587459    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc44455cf-tknzm" event={"ID":"f3bcb79d-3ca1-49f4-bbe3-19d10b620a45","Type":"ContainerDied","Data":"ae4e38f6f653ae4c3bbd460fe5be63827c58b73cd03d062cd5b8e23e317081d7"}
Oct 31 03:29:37.588003 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:37.587520    5906 scope.go:122] "RemoveContainer" containerID="47bb757ff7524da3958839b661189e02f3616fecb541e7212868624b8a8102c3"
Oct 31 03:29:37.588003 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:37.587479    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc44455cf-tknzm"
Oct 31 03:29:37.755848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:37.755816    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dc44455cf-tknzm"]
Oct 31 03:29:37.807633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:37.807589    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dc44455cf-tknzm"]
Oct 31 03:29:38.350289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:38.350257    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" path="/var/lib/kubelet/pods/f3bcb79d-3ca1-49f4-bbe3-19d10b620a45/volumes"
Oct 31 03:29:39.045376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:39.045333    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:39.890939 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:39.890906    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:41.649182 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:41.649143    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:41.784616 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:41.784568    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:29:42.763820 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:42.763774    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59c57d5c75-2rvq9" podUID="bace1d01-5c41-4521-91b4-7ce780044fbe" containerName="console" containerID="cri-o://5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd" gracePeriod=15
Oct 31 03:29:43.053091 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.053062    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:43.190870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.190837    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.190919    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191024    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191052    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191154 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191092    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191154 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191120    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191154    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191372 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191188    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") pod \"bace1d01-5c41-4521-91b4-7ce780044fbe\" (UID: \"bace1d01-5c41-4521-91b4-7ce780044fbe\") "
Oct 31 03:29:43.191543 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191512    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:43.191543 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191522    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:43.191679 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191522    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:43.191734 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191686    5906 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.191734 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191705    5906 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-oauth-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.191734 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191709    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:29:43.191734 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191721    5906 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-trusted-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.191935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.191751    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:43.193360 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.193338    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:43.193469 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.193453    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:43.193530 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.193498    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4" pod "bace1d01-5c41-4521-91b4-7ce780044fbe" (UID: "bace1d01-5c41-4521-91b4-7ce780044fbe"). InnerVolumeSpecName "kube-api-access-p5rd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:43.292255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.292152    5906 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.292255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.292178    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5rd4\" (UniqueName: \"kubernetes.io/projected/bace1d01-5c41-4521-91b4-7ce780044fbe-kube-api-access-p5rd4\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.292255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.292190    5906 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bace1d01-5c41-4521-91b4-7ce780044fbe-console-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.292255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.292200    5906 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bace1d01-5c41-4521-91b4-7ce780044fbe-tmp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.292255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.292222    5906 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bace1d01-5c41-4521-91b4-7ce780044fbe-console-oauth-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:43.379339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.379310    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7qbtw"
Oct 31 03:29:43.612083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.612000    5906 generic.go:356] "Generic (PLEG): container finished" podID="bace1d01-5c41-4521-91b4-7ce780044fbe" containerID="5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd" exitCode=0
Oct 31 03:29:43.612083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.612050    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59c57d5c75-2rvq9"
Oct 31 03:29:43.612083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.612061    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c57d5c75-2rvq9" event={"ID":"bace1d01-5c41-4521-91b4-7ce780044fbe","Type":"ContainerDied","Data":"5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd"}
Oct 31 03:29:43.612342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.612099    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59c57d5c75-2rvq9" event={"ID":"bace1d01-5c41-4521-91b4-7ce780044fbe","Type":"ContainerDied","Data":"350fe15d4147c9e837a3d28c387a55c00fc02df1cd580b25dba499c6ed0f34c9"}
Oct 31 03:29:43.612342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.612118    5906 scope.go:122] "RemoveContainer" containerID="5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd"
Oct 31 03:29:43.620858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.620839    5906 scope.go:122] "RemoveContainer" containerID="5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd"
Oct 31 03:29:43.621161 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:29:43.621141    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd\": container with ID starting with 5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd not found: ID does not exist" containerID="5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd"
Oct 31 03:29:43.621231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.621172    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd"} err="failed to get container status \"5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd\": rpc error: code = NotFound desc = could not find container \"5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd\": container with ID starting with 5c4c551418436f842db8e5ccebf2c125e9aeae2c41fc7a0edb7c8a2276869cfd not found: ID does not exist"
Oct 31 03:29:43.780123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.780094    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59c57d5c75-2rvq9"]
Oct 31 03:29:43.783903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:43.783875    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59c57d5c75-2rvq9"]
Oct 31 03:29:44.349538 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:29:44.349499    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bace1d01-5c41-4521-91b4-7ce780044fbe" path="/var/lib/kubelet/pods/bace1d01-5c41-4521-91b4-7ce780044fbe/volumes"
Oct 31 03:30:02.808951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:02.808909    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:02.828107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:02.828078    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:03.702393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:03.702367    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:06.381694 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:06.381653    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:30:14.159503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:14.159461    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:14.299426 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:14.299388    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:18.967462 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:18.967427    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:19.151368 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:19.151334    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:21.118747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.118714    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:30:21.119200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.119026    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="alertmanager" containerID="cri-o://cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f" gracePeriod=120
Oct 31 03:30:21.119200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.119099    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-web" containerID="cri-o://32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3" gracePeriod=120
Oct 31 03:30:21.119200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.119117    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="config-reloader" containerID="cri-o://016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b" gracePeriod=120
Oct 31 03:30:21.119200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.119144    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy" containerID="cri-o://db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f" gracePeriod=120
Oct 31 03:30:21.119200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.119173    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="prom-label-proxy" containerID="cri-o://da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0" gracePeriod=120
Oct 31 03:30:21.119498 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.119246    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-metric" containerID="cri-o://117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306" gracePeriod=120
Oct 31 03:30:21.757446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757414    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0" exitCode=0
Oct 31 03:30:21.757446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757440    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f" exitCode=0
Oct 31 03:30:21.757446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757446    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b" exitCode=0
Oct 31 03:30:21.757446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757452    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f" exitCode=0
Oct 31 03:30:21.757715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757483    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"}
Oct 31 03:30:21.757715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757548    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"}
Oct 31 03:30:21.757715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757575    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"}
Oct 31 03:30:21.757715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:21.757599    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"}
Oct 31 03:30:22.374327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.374297    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:22.536894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.536859    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.536919    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.536965    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537003    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537040    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537069    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537132    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537165    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537201    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537266    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537314    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537368    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.537609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.537415    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") pod \"91095503-a34e-402b-9017-84b2a5379894\" (UID: \"91095503-a34e-402b-9017-84b2a5379894\") "
Oct 31 03:30:22.538069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.538040    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:22.539225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.539186    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:30:22.539782 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.539758    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:22.540503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.540464    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "kube-api-access-lvrt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:30:22.540804 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.540760    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.541057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.540910    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:30:22.541057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.541022    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.541247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.541135    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.541460 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.541432    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:30:22.541637 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.541614    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.542142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.542118    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.544782 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.544759    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.551501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.551479    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config" pod "91095503-a34e-402b-9017-84b2a5379894" (UID: "91095503-a34e-402b-9017-84b2a5379894"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:22.638884 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638850    5906 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.638884 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638879    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvrt8\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-kube-api-access-lvrt8\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.638884 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638890    5906 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-cluster-tls-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638900    5906 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-web-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638908    5906 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638918    5906 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638928    5906 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/91095503-a34e-402b-9017-84b2a5379894-tls-assets\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638936    5906 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638946    5906 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-alertmanager-main-db\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638955    5906 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/91095503-a34e-402b-9017-84b2a5379894-config-out\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638963    5906 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638972    5906 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/91095503-a34e-402b-9017-84b2a5379894-secret-alertmanager-main-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.639118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.638981    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91095503-a34e-402b-9017-84b2a5379894-metrics-client-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:22.763987 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.763956    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306" exitCode=0
Oct 31 03:30:22.763987 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.763981    5906 generic.go:356] "Generic (PLEG): container finished" podID="91095503-a34e-402b-9017-84b2a5379894" containerID="32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3" exitCode=0
Oct 31 03:30:22.764174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.764032    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:22.764174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.764032    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"}
Oct 31 03:30:22.764174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.764100    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"}
Oct 31 03:30:22.764174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.764127    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"91095503-a34e-402b-9017-84b2a5379894","Type":"ContainerDied","Data":"65616deff14cbaa55492dae424605bf54124ec1fbac1f399eb49f71161fabdc9"}
Oct 31 03:30:22.764174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.764166    5906 scope.go:122] "RemoveContainer" containerID="da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"
Oct 31 03:30:22.772286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.772270    5906 scope.go:122] "RemoveContainer" containerID="117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"
Oct 31 03:30:22.779415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.779395    5906 scope.go:122] "RemoveContainer" containerID="db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"
Oct 31 03:30:22.786874 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.786856    5906 scope.go:122] "RemoveContainer" containerID="32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"
Oct 31 03:30:22.793596 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.793579    5906 scope.go:122] "RemoveContainer" containerID="016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"
Oct 31 03:30:22.800391 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.800371    5906 scope.go:122] "RemoveContainer" containerID="cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"
Oct 31 03:30:22.807271 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.807255    5906 scope.go:122] "RemoveContainer" containerID="e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"
Oct 31 03:30:22.813819 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.813801    5906 scope.go:122] "RemoveContainer" containerID="da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"
Oct 31 03:30:22.814075 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.814059    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0\": container with ID starting with da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0 not found: ID does not exist" containerID="da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"
Oct 31 03:30:22.814136 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814087    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"} err="failed to get container status \"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0\": rpc error: code = NotFound desc = could not find container \"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0\": container with ID starting with da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0 not found: ID does not exist"
Oct 31 03:30:22.814136 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814106    5906 scope.go:122] "RemoveContainer" containerID="117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"
Oct 31 03:30:22.814319 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.814301    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306\": container with ID starting with 117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306 not found: ID does not exist" containerID="117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"
Oct 31 03:30:22.814384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814330    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"} err="failed to get container status \"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306\": rpc error: code = NotFound desc = could not find container \"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306\": container with ID starting with 117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306 not found: ID does not exist"
Oct 31 03:30:22.814384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814353    5906 scope.go:122] "RemoveContainer" containerID="db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"
Oct 31 03:30:22.814594 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.814577    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f\": container with ID starting with db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f not found: ID does not exist" containerID="db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"
Oct 31 03:30:22.814641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814605    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"} err="failed to get container status \"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f\": rpc error: code = NotFound desc = could not find container \"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f\": container with ID starting with db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f not found: ID does not exist"
Oct 31 03:30:22.814641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814618    5906 scope.go:122] "RemoveContainer" containerID="32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"
Oct 31 03:30:22.814861 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.814844    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3\": container with ID starting with 32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3 not found: ID does not exist" containerID="32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"
Oct 31 03:30:22.814926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814869    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"} err="failed to get container status \"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3\": rpc error: code = NotFound desc = could not find container \"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3\": container with ID starting with 32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3 not found: ID does not exist"
Oct 31 03:30:22.814926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.814890    5906 scope.go:122] "RemoveContainer" containerID="016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"
Oct 31 03:30:22.815132 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.815115    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b\": container with ID starting with 016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b not found: ID does not exist" containerID="016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"
Oct 31 03:30:22.815180 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815137    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"} err="failed to get container status \"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b\": rpc error: code = NotFound desc = could not find container \"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b\": container with ID starting with 016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b not found: ID does not exist"
Oct 31 03:30:22.815180 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815150    5906 scope.go:122] "RemoveContainer" containerID="cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"
Oct 31 03:30:22.815384 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.815368    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f\": container with ID starting with cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f not found: ID does not exist" containerID="cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"
Oct 31 03:30:22.815427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815390    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"} err="failed to get container status \"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f\": rpc error: code = NotFound desc = could not find container \"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f\": container with ID starting with cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f not found: ID does not exist"
Oct 31 03:30:22.815427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815402    5906 scope.go:122] "RemoveContainer" containerID="e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"
Oct 31 03:30:22.815588 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:22.815573    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea\": container with ID starting with e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea not found: ID does not exist" containerID="e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"
Oct 31 03:30:22.815626 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815591    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"} err="failed to get container status \"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea\": rpc error: code = NotFound desc = could not find container \"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea\": container with ID starting with e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea not found: ID does not exist"
Oct 31 03:30:22.815626 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815604    5906 scope.go:122] "RemoveContainer" containerID="da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"
Oct 31 03:30:22.815796 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815780    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0"} err="failed to get container status \"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0\": rpc error: code = NotFound desc = could not find container \"da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0\": container with ID starting with da4462a71369779375cee342c2385e89a9ac07d9dcbbb9784fbdace799a6a3d0 not found: ID does not exist"
Oct 31 03:30:22.815796 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815795    5906 scope.go:122] "RemoveContainer" containerID="117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"
Oct 31 03:30:22.815980 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815965    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306"} err="failed to get container status \"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306\": rpc error: code = NotFound desc = could not find container \"117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306\": container with ID starting with 117097c664b616ffa96342bbcafc9044564df28ac787ac5f86079ab7ca71b306 not found: ID does not exist"
Oct 31 03:30:22.816021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.815979    5906 scope.go:122] "RemoveContainer" containerID="db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"
Oct 31 03:30:22.816167 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816148    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f"} err="failed to get container status \"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f\": rpc error: code = NotFound desc = could not find container \"db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f\": container with ID starting with db9082704502732d17dee3f7ea0ef62c25603c995350d89d00fd6e77b2974a8f not found: ID does not exist"
Oct 31 03:30:22.816234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816167    5906 scope.go:122] "RemoveContainer" containerID="32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"
Oct 31 03:30:22.816431 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816385    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3"} err="failed to get container status \"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3\": rpc error: code = NotFound desc = could not find container \"32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3\": container with ID starting with 32b5e453f2e9547383b85b3fb7a32de85db9a19280d69789cfa46a4f718579f3 not found: ID does not exist"
Oct 31 03:30:22.816431 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816399    5906 scope.go:122] "RemoveContainer" containerID="016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"
Oct 31 03:30:22.816570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816555    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b"} err="failed to get container status \"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b\": rpc error: code = NotFound desc = could not find container \"016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b\": container with ID starting with 016e7e262d14c68e7783e58890c5c42e3a9e36cc93620476ce3cec092cfc3b5b not found: ID does not exist"
Oct 31 03:30:22.816609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816570    5906 scope.go:122] "RemoveContainer" containerID="cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"
Oct 31 03:30:22.816801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816781    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f"} err="failed to get container status \"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f\": rpc error: code = NotFound desc = could not find container \"cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f\": container with ID starting with cf5163f96f6a493532d203b5e472abb16d59a46a4b7b6c244a31301ab5c0558f not found: ID does not exist"
Oct 31 03:30:22.816844 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816802    5906 scope.go:122] "RemoveContainer" containerID="e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"
Oct 31 03:30:22.816978 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.816960    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea"} err="failed to get container status \"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea\": rpc error: code = NotFound desc = could not find container \"e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea\": container with ID starting with e483215bb580df8231cec30cb0076becaa9f649b1020cb88ef08e85d696ffaea not found: ID does not exist"
Oct 31 03:30:22.936107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.936074    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:30:22.943376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.941317    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:30:22.987521 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987486    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:30:22.987806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987793    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-web"
Oct 31 03:30:22.987849 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987807    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-web"
Oct 31 03:30:22.987849 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987820    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy"
Oct 31 03:30:22.987849 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987825    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy"
Oct 31 03:30:22.987849 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987837    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="alertmanager"
Oct 31 03:30:22.987849 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987842    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="alertmanager"
Oct 31 03:30:22.987849 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987850    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="init-config-reloader"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987855    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="init-config-reloader"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987861    5906 cpu_manager.go:414] "containerMap: removing container" podUID="bace1d01-5c41-4521-91b4-7ce780044fbe" containerName="console"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987866    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bace1d01-5c41-4521-91b4-7ce780044fbe" containerName="console"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987877    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-metric"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987882    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-metric"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987889    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" containerName="console"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987893    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" containerName="console"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987899    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="config-reloader"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987903    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="config-reloader"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987914    5906 cpu_manager.go:414] "containerMap: removing container" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="prom-label-proxy"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987918    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="prom-label-proxy"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987963    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987973    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="alertmanager"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987979    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="init-config-reloader"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987985    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="bace1d01-5c41-4521-91b4-7ce780044fbe" containerName="console"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987992    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-metric"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.987997    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3bcb79d-3ca1-49f4-bbe3-19d10b620a45" containerName="console"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.988002    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="config-reloader"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.988008    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="prom-label-proxy"
Oct 31 03:30:22.988021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.988014    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="91095503-a34e-402b-9017-84b2a5379894" containerName="kube-rbac-proxy-web"
Oct 31 03:30:22.991667 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:22.991647    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.046890    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.046975    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047042    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047086    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047131    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047252    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047321    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-njklp\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047398    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 31 03:30:23.047602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047585    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 31 03:30:23.048216 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.047895    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 31 03:30:23.098307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.098275    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:30:23.144383 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144344    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144383 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144381    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-config-out\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144402    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144549    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-web-config\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144584    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144646    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vz7c\" (UniqueName: \"kubernetes.io/projected/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-kube-api-access-4vz7c\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144679    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144702    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144774    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144826    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-config-volume\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144845    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144992 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144870    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.144992 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.144886    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246371 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246334    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-config-volume\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246371 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246375    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246411    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246434    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246481    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246511    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-config-out\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246539    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246566    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-web-config\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246591    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.246636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246623    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vz7c\" (UniqueName: \"kubernetes.io/projected/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-kube-api-access-4vz7c\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.247122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246652    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.247122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246687    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.247122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246717    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.247122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.246807    5906 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 03:30:23.247122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.247077    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.248073 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.248040    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.248073 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.248061    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.249759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.249709    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.249759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.249720    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-config-volume\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.250016 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.249979    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.250222 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.250186    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.250278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.250197    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-config-out\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.250422 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.250404    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.250725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.250706    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.251051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.251034    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-web-config\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.251443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.251421    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.310198 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.310123    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vz7c\" (UniqueName: \"kubernetes.io/projected/869b8227-2fd2-45e9-a10a-9bf27ba98d3f-kube-api-access-4vz7c\") pod \"alertmanager-main-0\" (UID: \"869b8227-2fd2-45e9-a10a-9bf27ba98d3f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:23.609592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:23.609491    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 31 03:30:24.081809 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:30:24.081778    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869b8227_2fd2_45e9_a10a_9bf27ba98d3f.slice/crio-6478b2db4f55bd91c7b99c52591ff91e3e0ed02670a2b88e2953517d974e253e WatchSource:0}: Error finding container 6478b2db4f55bd91c7b99c52591ff91e3e0ed02670a2b88e2953517d974e253e: Status 404 returned error can't find the container with id 6478b2db4f55bd91c7b99c52591ff91e3e0ed02670a2b88e2953517d974e253e
Oct 31 03:30:24.082776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:24.082756    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 31 03:30:24.348986 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:24.348909    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91095503-a34e-402b-9017-84b2a5379894" path="/var/lib/kubelet/pods/91095503-a34e-402b-9017-84b2a5379894/volumes"
Oct 31 03:30:24.774045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:24.774016    5906 generic.go:356] "Generic (PLEG): container finished" podID="869b8227-2fd2-45e9-a10a-9bf27ba98d3f" containerID="140bcfe981c090ceea43bd01bbca2b0da0e8ba404b4685a88fb749082a90d8b3" exitCode=0
Oct 31 03:30:24.774440 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:24.774062    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerDied","Data":"140bcfe981c090ceea43bd01bbca2b0da0e8ba404b4685a88fb749082a90d8b3"}
Oct 31 03:30:24.774440 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:24.774120    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"6478b2db4f55bd91c7b99c52591ff91e3e0ed02670a2b88e2953517d974e253e"}
Oct 31 03:30:25.780782 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:25.780738    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"14659ab429dd072cf4a9b216587ef5f6dd80f3c2fa28745f5db1d0566973d05f"}
Oct 31 03:30:25.781178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:25.780820    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"aa33d52ee87ac35d621c9d84818fa54033eab54ed2e5fe76149967c57b360e90"}
Oct 31 03:30:25.781178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:25.780862    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"03b55cd6d32e5c843b2fe21ddbbe349298e6fb742567f87a43ad6575bfcd85cf"}
Oct 31 03:30:25.781178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:25.780902    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"ec2d359d228f5c5c9312bad57a9456dacef3047819b1f1b13ae121f9d3c230a2"}
Oct 31 03:30:25.781178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:25.780939    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"ce0a52cd22165854b8303202884fa8261a797246138ba2e35ef5574bbf977889"}
Oct 31 03:30:25.781178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:25.780972    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"869b8227-2fd2-45e9-a10a-9bf27ba98d3f","Type":"ContainerStarted","Data":"5d6e2318d54ed9f106b02f6112b4efb76f94cb99c0c82c200ceae292949cd6c9"}
Oct 31 03:30:26.841088 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.841011    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.8279390620000004 podStartE2EDuration="4.840990442s" totalImagesPullingTime="0s" totalInitContainerRuntime="13.05138ms" isStatefulPod=false podCreationTimestamp="2026-10-31 03:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:30:25.915964965 +0000 UTC m=+75.749825276" watchObservedRunningTime="2026-10-31 03:30:26.840990442 +0000 UTC m=+76.674850747"
Oct 31 03:30:26.842559 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.842532    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:30:26.843071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.843041    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="prometheus" containerID="cri-o://925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616" gracePeriod=600
Oct 31 03:30:26.843333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.843302    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy" containerID="cri-o://fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc" gracePeriod=600
Oct 31 03:30:26.843447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.843329    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-thanos" containerID="cri-o://6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507" gracePeriod=600
Oct 31 03:30:26.843447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.843354    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="thanos-sidecar" containerID="cri-o://a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd" gracePeriod=600
Oct 31 03:30:26.843563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.843481    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-web" containerID="cri-o://f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8" gracePeriod=600
Oct 31 03:30:26.844101 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:26.843657    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="config-reloader" containerID="cri-o://4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d" gracePeriod=600
Oct 31 03:30:27.099469 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.099394    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:27.186086 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186053    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186086 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186096    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186121    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186151    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186169    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186194    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186227    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186262    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186283    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186328    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186357    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186387    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186444    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186478    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186513    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186547    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186578    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186610    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186681    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.186836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186718    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") pod \"a647986f-3521-49de-950c-fb705df308ab\" (UID: \"a647986f-3521-49de-950c-fb705df308ab\") "
Oct 31 03:30:27.187327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.186904    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.187327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.187068    5906 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.187327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.187117    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.188826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.187530    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.188826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.187766    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.188826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.188120    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.188826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.188169    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.189033 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.188852    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:30:27.189033 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.188867    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:30:27.190311 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.190283    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.190663 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.190634    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.190785 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.190756    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.190904 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.190874    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.191071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.191015    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "kube-api-access-vs47r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:30:27.191071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.191022    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.191259 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.191195    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:30:27.191329 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.191258    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.191570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.191541    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:30:27.191808 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.191788    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.192531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.192510    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.201499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.201475    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config" pod "a647986f-3521-49de-950c-fb705df308ab" (UID: "a647986f-3521-49de-950c-fb705df308ab"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:30:27.288515 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288483    5906 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-grpc-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288515 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288513    5906 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288524    5906 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288534    5906 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288544    5906 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288553    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs47r\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-kube-api-access-vs47r\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288562    5906 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288571    5906 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-kube-rbac-proxy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288582    5906 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288590    5906 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-metrics-client-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288601    5906 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-web-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288611    5906 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288620    5906 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-prometheus-k8s-db\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288628    5906 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288637    5906 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-metrics-client-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288647    5906 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a647986f-3521-49de-950c-fb705df308ab-config-out\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288656    5906 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a647986f-3521-49de-950c-fb705df308ab-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288666    5906 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a647986f-3521-49de-950c-fb705df308ab-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.288711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.288674    5906 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a647986f-3521-49de-950c-fb705df308ab-tls-assets\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:30:27.792452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792417    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507" exitCode=0
Oct 31 03:30:27.792452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792448    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc" exitCode=0
Oct 31 03:30:27.792452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792457    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8" exitCode=0
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792466    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd" exitCode=0
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792474    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d" exitCode=0
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792482    5906 generic.go:356] "Generic (PLEG): container finished" podID="a647986f-3521-49de-950c-fb705df308ab" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616" exitCode=0
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792520    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792512    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"}
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792664    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"}
Oct 31 03:30:27.792696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792698    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"}
Oct 31 03:30:27.792918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792723    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"}
Oct 31 03:30:27.792918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792729    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.792918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792747    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"}
Oct 31 03:30:27.792918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792772    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"}
Oct 31 03:30:27.792918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.792795    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a647986f-3521-49de-950c-fb705df308ab","Type":"ContainerDied","Data":"15e355a67b6192a8c5775802ef0cf89dd0652b3d03545cc61bfdb26d10138e8e"}
Oct 31 03:30:27.801186 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.801163    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.808450 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.808431    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.815636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.815614    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.822726 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.822708    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.829595 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.829567    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.837864 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.837843    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.844867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.844676    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.845146 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.844945    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.845146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.844972    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"} err="failed to get container status \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist"
Oct 31 03:30:27.845146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.844989    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.845274 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.845226    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.845274 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.845258    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"} err="failed to get container status \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist"
Oct 31 03:30:27.845342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.845274    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.845545 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.845523    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.845592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.845554    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"} err="failed to get container status \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist"
Oct 31 03:30:27.845592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.845571    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.845801 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.845784    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.845846 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.845809    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"} err="failed to get container status \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist"
Oct 31 03:30:27.845846 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.845823    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.846037 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.846022    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.846076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846041    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"} err="failed to get container status \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist"
Oct 31 03:30:27.846076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846055    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.846282 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.846267    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.846329 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846286    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"} err="failed to get container status \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist"
Oct 31 03:30:27.846329 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846298    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.846515 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:27.846493    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.846554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846523    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"} err="failed to get container status \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist"
Oct 31 03:30:27.846554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846543    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.846738 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846723    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"} err="failed to get container status \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist"
Oct 31 03:30:27.846778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846738    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.846933 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846913    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"} err="failed to get container status \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist"
Oct 31 03:30:27.846987 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.846934    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.847274 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847248    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"} err="failed to get container status \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist"
Oct 31 03:30:27.847351 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847274    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.847554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847538    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"} err="failed to get container status \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist"
Oct 31 03:30:27.847600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847555    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.847772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847754    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"} err="failed to get container status \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist"
Oct 31 03:30:27.847817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847772    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.847959 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847944    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"} err="failed to get container status \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist"
Oct 31 03:30:27.848003 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.847959    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.848192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848176    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"} err="failed to get container status \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist"
Oct 31 03:30:27.848192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848191    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.848445 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848423    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"} err="failed to get container status \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist"
Oct 31 03:30:27.848491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848447    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.848699 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848679    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"} err="failed to get container status \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist"
Oct 31 03:30:27.848770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848700    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.848926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848909    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"} err="failed to get container status \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist"
Oct 31 03:30:27.848976 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.848926    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.849132 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849117    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"} err="failed to get container status \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist"
Oct 31 03:30:27.849175 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849131    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.849372 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849352    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"} err="failed to get container status \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist"
Oct 31 03:30:27.849444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849372    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.849598 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849579    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"} err="failed to get container status \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist"
Oct 31 03:30:27.849652 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849599    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.849811 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849792    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"} err="failed to get container status \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist"
Oct 31 03:30:27.849863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.849811    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.850027 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850006    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"} err="failed to get container status \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist"
Oct 31 03:30:27.850071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850028    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.850238 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850217    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"} err="failed to get container status \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist"
Oct 31 03:30:27.850315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850240    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.850458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850436    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"} err="failed to get container status \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist"
Oct 31 03:30:27.850513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850458    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.850649 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850633    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"} err="failed to get container status \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist"
Oct 31 03:30:27.850687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850650    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.850838 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850820    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"} err="failed to get container status \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist"
Oct 31 03:30:27.850882 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.850838    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.851071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851056    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"} err="failed to get container status \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist"
Oct 31 03:30:27.851133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851072    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.851332 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851313    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"} err="failed to get container status \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist"
Oct 31 03:30:27.851406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851332    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.851579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851560    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"} err="failed to get container status \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist"
Oct 31 03:30:27.851631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851580    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.851834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851814    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"} err="failed to get container status \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist"
Oct 31 03:30:27.851905 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.851835    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.852064 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852048    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"} err="failed to get container status \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist"
Oct 31 03:30:27.852112 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852065    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.852271 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852257    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"} err="failed to get container status \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist"
Oct 31 03:30:27.852320 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852271    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.852483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852466    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"} err="failed to get container status \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist"
Oct 31 03:30:27.852528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852483    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.852692 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852676    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"} err="failed to get container status \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist"
Oct 31 03:30:27.852730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852692    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.852910 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852893    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"} err="failed to get container status \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist"
Oct 31 03:30:27.852954 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.852910    5906 scope.go:122] "RemoveContainer" containerID="6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"
Oct 31 03:30:27.853104 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853088    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507"} err="failed to get container status \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": rpc error: code = NotFound desc = could not find container \"6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507\": container with ID starting with 6251ef1ac436df9c3642009c9697395827166ade4d3143592a0a1bcc916ea507 not found: ID does not exist"
Oct 31 03:30:27.853148 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853104    5906 scope.go:122] "RemoveContainer" containerID="fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"
Oct 31 03:30:27.853344 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853326    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc"} err="failed to get container status \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": rpc error: code = NotFound desc = could not find container \"fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc\": container with ID starting with fbf0da86698ca419242bfa1fe7adc2ca0aac5744c7b50bee6c8e7e390a58cffc not found: ID does not exist"
Oct 31 03:30:27.853403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853343    5906 scope.go:122] "RemoveContainer" containerID="f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"
Oct 31 03:30:27.853577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853559    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8"} err="failed to get container status \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": rpc error: code = NotFound desc = could not find container \"f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8\": container with ID starting with f394d5d04fd5dadfab14c059ca11704dfc141b94631d1b2dbfd4d1424ccfbab8 not found: ID does not exist"
Oct 31 03:30:27.853577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853575    5906 scope.go:122] "RemoveContainer" containerID="a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"
Oct 31 03:30:27.853805 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853784    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd"} err="failed to get container status \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": rpc error: code = NotFound desc = could not find container \"a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd\": container with ID starting with a4f0d8c839cde091fab9798a1b3fcb76212c53b6bb5a40a85a1017373cdd48cd not found: ID does not exist"
Oct 31 03:30:27.853864 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.853806    5906 scope.go:122] "RemoveContainer" containerID="4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"
Oct 31 03:30:27.854048 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.854030    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d"} err="failed to get container status \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": rpc error: code = NotFound desc = could not find container \"4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d\": container with ID starting with 4d2c864c2ebe06a7421dc03f602d2c8241b39dcac0920c2e1ca17e768a82f95d not found: ID does not exist"
Oct 31 03:30:27.854108 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.854048    5906 scope.go:122] "RemoveContainer" containerID="925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"
Oct 31 03:30:27.854280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.854264    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616"} err="failed to get container status \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": rpc error: code = NotFound desc = could not find container \"925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616\": container with ID starting with 925efcc14ab2d81142ffcf82cee6c6a69a7700031ee203ef35d334efcf741616 not found: ID does not exist"
Oct 31 03:30:27.854280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.854279    5906 scope.go:122] "RemoveContainer" containerID="380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"
Oct 31 03:30:27.854496 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.854476    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88"} err="failed to get container status \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": rpc error: code = NotFound desc = could not find container \"380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88\": container with ID starting with 380f4b01c257f0fffe18aaee7c47d769aca3e27ac6d41c5463ec490cc0482b88 not found: ID does not exist"
Oct 31 03:30:27.965273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:27.965241    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:30:28.016843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.016811    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:30:28.017283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017266    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:30:28.017549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017537    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="thanos-sidecar"
Oct 31 03:30:28.017586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017550    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="thanos-sidecar"
Oct 31 03:30:28.017586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017564    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="config-reloader"
Oct 31 03:30:28.017586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017570    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="config-reloader"
Oct 31 03:30:28.017586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017576    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy"
Oct 31 03:30:28.017586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017581    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017594    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="init-config-reloader"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017599    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="init-config-reloader"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017607    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-web"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017611    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-web"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017626    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-thanos"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017631    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-thanos"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017640    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="prometheus"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017645    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="prometheus"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017688    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="thanos-sidecar"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017698    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="config-reloader"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017703    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017711    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-web"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017719    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="kube-rbac-proxy-thanos"
Oct 31 03:30:28.017725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.017724    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a647986f-3521-49de-950c-fb705df308ab" containerName="prometheus"
Oct 31 03:30:28.021577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.021559    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.102867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.102786    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 31 03:30:28.102867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.102846    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 31 03:30:28.103543 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103517    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8641tjbr9hbo8\""
Oct 31 03:30:28.103662 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103555    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 31 03:30:28.103752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103734    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 31 03:30:28.103752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103747    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 31 03:30:28.103938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103756    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 31 03:30:28.103938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103778    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 31 03:30:28.103938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103856    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 31 03:30:28.103938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103880    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 31 03:30:28.103938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103896    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 31 03:30:28.103938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103926    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 31 03:30:28.104150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.103960    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ktggl\""
Oct 31 03:30:28.155829 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.155795    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 31 03:30:28.156055 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.155876    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 31 03:30:28.156351 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.156331    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 31 03:30:28.161019 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.157637    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:30:28.195965 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.195932    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.195965 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.195967    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.195991    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/072a8b97-c371-403e-981b-1c13e6d0080d-config-out\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196074    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196116    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196136    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196172    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196233    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196283    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196313    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196344    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg6jr\" (UniqueName: \"kubernetes.io/projected/072a8b97-c371-403e-981b-1c13e6d0080d-kube-api-access-mg6jr\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196393    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-config\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196425    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196441    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196458    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196485    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196518    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/072a8b97-c371-403e-981b-1c13e6d0080d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196541    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196746 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196591    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-web-config\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.196746 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.196613    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.297553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297517    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.297553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297558    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.297785 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297581    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.297785 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297607    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.297785 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297636    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.297785 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297668    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg6jr\" (UniqueName: \"kubernetes.io/projected/072a8b97-c371-403e-981b-1c13e6d0080d-kube-api-access-mg6jr\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297827    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-config\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297891    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297919    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297947    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.297972    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298008    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298027    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/072a8b97-c371-403e-981b-1c13e6d0080d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298053    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298081    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-web-config\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298104    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298156    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298177    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298196    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/072a8b97-c371-403e-981b-1c13e6d0080d-config-out\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298243    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298262    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298331    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298503    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.298963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.298600    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.300502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.299287    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.300502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.299287    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.300996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.300869    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.300996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.300871    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.300996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.300961    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.300996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.300960    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-config\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.302155 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.302126    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.302919 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.302822    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.303226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.303154    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/072a8b97-c371-403e-981b-1c13e6d0080d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.303366 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.303308    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.303478 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.303454    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/072a8b97-c371-403e-981b-1c13e6d0080d-config-out\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.303538 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.303460    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/072a8b97-c371-403e-981b-1c13e6d0080d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.303594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.303560    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.303594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.303584    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.304382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.304363    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/072a8b97-c371-403e-981b-1c13e6d0080d-web-config\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.349316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.349287    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a647986f-3521-49de-950c-fb705df308ab" path="/var/lib/kubelet/pods/a647986f-3521-49de-950c-fb705df308ab/volumes"
Oct 31 03:30:28.361153 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.361082    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg6jr\" (UniqueName: \"kubernetes.io/projected/072a8b97-c371-403e-981b-1c13e6d0080d-kube-api-access-mg6jr\") pod \"prometheus-k8s-0\" (UID: \"072a8b97-c371-403e-981b-1c13e6d0080d\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:28.639800 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:28.639699    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:29.113266 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:30:29.113234    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072a8b97_c371_403e_981b_1c13e6d0080d.slice/crio-6b0a1da52764e5c0ff323e1902e25a1664c131af01bd14c032cd60bde0883635 WatchSource:0}: Error finding container 6b0a1da52764e5c0ff323e1902e25a1664c131af01bd14c032cd60bde0883635: Status 404 returned error can't find the container with id 6b0a1da52764e5c0ff323e1902e25a1664c131af01bd14c032cd60bde0883635
Oct 31 03:30:29.115249 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:29.115228    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 31 03:30:29.802236 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:29.802187    5906 generic.go:356] "Generic (PLEG): container finished" podID="072a8b97-c371-403e-981b-1c13e6d0080d" containerID="d3d5a77730146c26034154a71e389a13bc86832fd4d2e825199dfb10d9d523d0" exitCode=0
Oct 31 03:30:29.802415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:29.802273    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerDied","Data":"d3d5a77730146c26034154a71e389a13bc86832fd4d2e825199dfb10d9d523d0"}
Oct 31 03:30:29.802415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:29.802340    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"6b0a1da52764e5c0ff323e1902e25a1664c131af01bd14c032cd60bde0883635"}
Oct 31 03:30:30.226295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.226260    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d9bcc8bff-lrgdt"]
Oct 31 03:30:30.228625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.228608    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.286415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.286385    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9bcc8bff-lrgdt"]
Oct 31 03:30:30.415901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.415816    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.415901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.415852    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.416098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.415939    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.416098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.415973    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.416098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.415991    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.416098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.416012    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.416098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.416036    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76mxk\" (UniqueName: \"kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.416348 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.416121    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.509491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.509460    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:30.517562 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517534    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517721 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517567    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517736    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517799    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517829    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517862    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517889    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76mxk\" (UniqueName: \"kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.517994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.517920    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.518262 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.518241    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.518356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.518343    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.518710 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.518689    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.518758 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.518728    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.519045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.519020    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.520159 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.520138    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.520326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.520309    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.582278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.582240    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76mxk\" (UniqueName: \"kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk\") pod \"console-5d9bcc8bff-lrgdt\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") " pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.647831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.647786    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:30.812045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.812009    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"5c932c8dadf8ce7f8e1bc96cf63f92de90998e82b7a9aa99707958e761fc9674"}
Oct 31 03:30:30.812193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.812098    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"90abe1f1b9d32c936979905aff7990dfe6f0ed534b3f82f22e39d0d3b0012628"}
Oct 31 03:30:30.812193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.812126    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"2fda83e94e1e4feab59810bef0e553be1d23b80552eeac04ad5539b087904760"}
Oct 31 03:30:30.812193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.812151    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"d57e4f49cecbc07e9a8199c9f583641e8e9063474a4e6ce11601b0b64f7b2fb3"}
Oct 31 03:30:30.812193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.812176    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"383d102b50df1c507fc89f5d7e63dbebfd484d2b984aa845c0979451500ce4ac"}
Oct 31 03:30:30.812360 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.812198    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"072a8b97-c371-403e-981b-1c13e6d0080d","Type":"ContainerStarted","Data":"063fe2b379cd50cc3435f4a8744a0fc9fac6e465c5b5568cce9aa98dedbac96a"}
Oct 31 03:30:30.838850 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.838809    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:30.933421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.933388    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:30.948540 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:30.948495    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.9323486709999997 podStartE2EDuration="3.948483334s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.134663ms" isStatefulPod=false podCreationTimestamp="2026-10-31 03:30:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:30:30.947745854 +0000 UTC m=+80.781606159" watchObservedRunningTime="2026-10-31 03:30:30.948483334 +0000 UTC m=+80.782343639"
Oct 31 03:30:31.076365 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:31.076268    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:31.208013 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:30:31.207981    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b782f74_31e7_4fe9_809d_d0400a823e76.slice/crio-d4f12df3b8ea5131f9e4684c9e6245aec85fdc17347e410f696dff2e41a787eb WatchSource:0}: Error finding container d4f12df3b8ea5131f9e4684c9e6245aec85fdc17347e410f696dff2e41a787eb: Status 404 returned error can't find the container with id d4f12df3b8ea5131f9e4684c9e6245aec85fdc17347e410f696dff2e41a787eb
Oct 31 03:30:31.208622 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:31.208598    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d9bcc8bff-lrgdt"]
Oct 31 03:30:31.817578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:31.817540    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9bcc8bff-lrgdt" event={"ID":"1b782f74-31e7-4fe9-809d-d0400a823e76","Type":"ContainerStarted","Data":"3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de"}
Oct 31 03:30:31.817947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:31.817597    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9bcc8bff-lrgdt" event={"ID":"1b782f74-31e7-4fe9-809d-d0400a823e76","Type":"ContainerStarted","Data":"d4f12df3b8ea5131f9e4684c9e6245aec85fdc17347e410f696dff2e41a787eb"}
Oct 31 03:30:31.940504 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:31.940454    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5d9bcc8bff-lrgdt" podStartSLOduration=1.9404382930000001 podStartE2EDuration="1.940438293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:30:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:30:31.93998873 +0000 UTC m=+81.773849035" watchObservedRunningTime="2026-10-31 03:30:31.940438293 +0000 UTC m=+81.774298666"
Oct 31 03:30:33.640344 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:33.640303    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:30:34.651294 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:34.651260    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:30:36.380311 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:30:36.380271    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:30:40.838981 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:40.838938    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:40.839421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:40.839004    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:40.843894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:40.843868    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:40.858681 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:40.858646    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:30:41.119313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:41.119232    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db55f7dc6-jpb5c"]
Oct 31 03:30:50.539521 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:30:50.539435    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:06.145467 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.145400    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-6db55f7dc6-jpb5c" podUID="3dbbc148-54de-4755-ba0b-8dee372b9917" containerName="console" containerID="cri-o://a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751" gracePeriod=15
Oct 31 03:31:06.380132 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:31:06.380091    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:31:06.408376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.408347    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:31:06.532852 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.532820    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.532888    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.532922    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.532970    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533018    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533049    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533103    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533159    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") pod \"3dbbc148-54de-4755-ba0b-8dee372b9917\" (UID: \"3dbbc148-54de-4755-ba0b-8dee372b9917\") "
Oct 31 03:31:06.533393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533309    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:31:06.533393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533344    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:31:06.533490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533429    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:31:06.533623 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533604    5906 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-oauth-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.533670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533631    5906 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-console-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.533670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533646    5906 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3dbbc148-54de-4755-ba0b-8dee372b9917-tmp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.533670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533657    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:31:06.533817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.533685    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:31:06.535314 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.535288    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "kube-api-access-49jch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:31:06.535555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.535539    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:31:06.535648 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.535626    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert" pod "3dbbc148-54de-4755-ba0b-8dee372b9917" (UID: "3dbbc148-54de-4755-ba0b-8dee372b9917"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:31:06.634418 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.634381    5906 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.634418 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.634413    5906 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.634418 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.634423    5906 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3dbbc148-54de-4755-ba0b-8dee372b9917-console-oauth-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.634655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.634432    5906 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3dbbc148-54de-4755-ba0b-8dee372b9917-trusted-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.634655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.634445    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49jch\" (UniqueName: \"kubernetes.io/projected/3dbbc148-54de-4755-ba0b-8dee372b9917-kube-api-access-49jch\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:31:06.959645 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.959609    5906 generic.go:356] "Generic (PLEG): container finished" podID="3dbbc148-54de-4755-ba0b-8dee372b9917" containerID="a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751" exitCode=0
Oct 31 03:31:06.959825 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.959657    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6db55f7dc6-jpb5c"
Oct 31 03:31:06.959825 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.959693    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db55f7dc6-jpb5c" event={"ID":"3dbbc148-54de-4755-ba0b-8dee372b9917","Type":"ContainerDied","Data":"a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751"}
Oct 31 03:31:06.959825 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.959742    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6db55f7dc6-jpb5c" event={"ID":"3dbbc148-54de-4755-ba0b-8dee372b9917","Type":"ContainerDied","Data":"4bd18deee211dd6dc6597de097aa941e0501ad95d9666191967976ef8e3511c5"}
Oct 31 03:31:06.959825 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.959762    5906 scope.go:122] "RemoveContainer" containerID="a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751"
Oct 31 03:31:06.968369 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.968348    5906 scope.go:122] "RemoveContainer" containerID="a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751"
Oct 31 03:31:06.968684 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:31:06.968664    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751\": container with ID starting with a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751 not found: ID does not exist" containerID="a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751"
Oct 31 03:31:06.968748 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:06.968694    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751"} err="failed to get container status \"a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751\": rpc error: code = NotFound desc = could not find container \"a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751\": container with ID starting with a6758915fb2e96bf425658d7622567fb45a17d5110d2a75a4069663eb44e7751 not found: ID does not exist"
Oct 31 03:31:07.128234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:07.128187    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6db55f7dc6-jpb5c"]
Oct 31 03:31:07.179806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:07.179776    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6db55f7dc6-jpb5c"]
Oct 31 03:31:08.349562 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:08.349525    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3dbbc148-54de-4755-ba0b-8dee372b9917" path="/var/lib/kubelet/pods/3dbbc148-54de-4755-ba0b-8dee372b9917/volumes"
Oct 31 03:31:13.241406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:13.241373    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:14.487837 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:14.487803    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:28.640295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:28.640262    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:31:28.656080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:28.656053    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:31:29.057568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:29.057542    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 31 03:31:36.380379 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:31:36.380336    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:31:36.680375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:36.680287    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:36.819497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:36.819466    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:37.585777 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.585743    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-56756d6f48-64glv"]
Oct 31 03:31:37.586290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.586235    5906 cpu_manager.go:414] "containerMap: removing container" podUID="3dbbc148-54de-4755-ba0b-8dee372b9917" containerName="console"
Oct 31 03:31:37.586290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.586253    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dbbc148-54de-4755-ba0b-8dee372b9917" containerName="console"
Oct 31 03:31:37.586429 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.586348    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dbbc148-54de-4755-ba0b-8dee372b9917" containerName="console"
Oct 31 03:31:37.590623 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.590602    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.697896 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.697866    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56756d6f48-64glv"]
Oct 31 03:31:37.709773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.709736    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-oauth-serving-cert\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.709903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.709783    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-service-ca\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.709903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.709811    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-trusted-ca-bundle\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.709998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.709912    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-serving-cert\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.709998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.709974    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-tmp\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.709998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.709994    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8kzv\" (UniqueName: \"kubernetes.io/projected/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-kube-api-access-d8kzv\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.710148 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.710092    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-oauth-config\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.710148 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.710133    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-config\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811042    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-tmp\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811082    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8kzv\" (UniqueName: \"kubernetes.io/projected/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-kube-api-access-d8kzv\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811118    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-oauth-config\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811139    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-config\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811168    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-oauth-serving-cert\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811339    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-service-ca\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811375    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-trusted-ca-bundle\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811405    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-serving-cert\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811611 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811521    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-tmp\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.811940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811912    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-oauth-serving-cert\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.812057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.811965    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-config\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.812057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.812001    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-service-ca\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.812417 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.812400    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-trusted-ca-bundle\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.813719 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.813700    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-oauth-config\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.813813 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.813788    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-console-serving-cert\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.873443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.873345    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:37.874815 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.874792    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8kzv\" (UniqueName: \"kubernetes.io/projected/f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1-kube-api-access-d8kzv\") pod \"console-56756d6f48-64glv\" (UID: \"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1\") " pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:37.901633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:37.901597    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:38.018583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:38.018554    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:38.272090 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:31:38.272055    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c4219f_4ac9_4a45_bbdc_9bf68a965ce1.slice/crio-b467a6c8a3655eac077b5617665303e5ab095b2bfaf2ed43fae643730fade217 WatchSource:0}: Error finding container b467a6c8a3655eac077b5617665303e5ab095b2bfaf2ed43fae643730fade217: Status 404 returned error can't find the container with id b467a6c8a3655eac077b5617665303e5ab095b2bfaf2ed43fae643730fade217
Oct 31 03:31:38.272733 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:38.272702    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56756d6f48-64glv"]
Oct 31 03:31:39.076313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:39.076280    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56756d6f48-64glv" event={"ID":"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1","Type":"ContainerStarted","Data":"b0cd115d03dfc689032a666619aded52da6b2cb26143053ce6e5f0c6429f0048"}
Oct 31 03:31:39.076692 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:39.076329    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56756d6f48-64glv" event={"ID":"f2c4219f-4ac9-4a45-bbdc-9bf68a965ce1","Type":"ContainerStarted","Data":"b467a6c8a3655eac077b5617665303e5ab095b2bfaf2ed43fae643730fade217"}
Oct 31 03:31:39.198199 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:39.198140    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-56756d6f48-64glv" podStartSLOduration=2.198124719 podStartE2EDuration="2.198124719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:31:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:31:39.197954733 +0000 UTC m=+149.031815084" watchObservedRunningTime="2026-10-31 03:31:39.198124719 +0000 UTC m=+149.031985027"
Oct 31 03:31:39.222577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:39.222542    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:39.358839 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:39.358753    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:47.902339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:47.902288    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:47.902339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:47.902352    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:47.906929 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:47.906906    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:48.113100 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:48.113072    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56756d6f48-64glv"
Oct 31 03:31:48.263479 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:48.263445    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d9bcc8bff-lrgdt"]
Oct 31 03:31:49.334259 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:49.334219    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:49.476585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:49.476553    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:56.656374 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:56.656335    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:31:56.796152 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:31:56.796121    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:32:06.379576 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:32:06.379527    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:32:13.283408 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.283343    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-5d9bcc8bff-lrgdt" podUID="1b782f74-31e7-4fe9-809d-d0400a823e76" containerName="console" containerID="cri-o://3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de" gracePeriod=15
Oct 31 03:32:13.525494 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.525466    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:32:13.530414 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530393    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530532 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530448    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530532 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530485    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530532 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530508    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530544    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk\" (UniqueName: \"kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530571    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530589    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.530690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530623    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp\") pod \"1b782f74-31e7-4fe9-809d-d0400a823e76\" (UID: \"1b782f74-31e7-4fe9-809d-d0400a823e76\") "
Oct 31 03:32:13.531063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530943    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:32:13.531063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530991    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:32:13.531063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.530955    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:32:13.531292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.531056    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:32:13.531292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.531084    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:32:13.532746 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.532720    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:32:13.532828 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.532800    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "kube-api-access-76mxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:32:13.532870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.532818    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert" pod "1b782f74-31e7-4fe9-809d-d0400a823e76" (UID: "1b782f74-31e7-4fe9-809d-d0400a823e76"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:32:13.631171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631131    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76mxk\" (UniqueName: \"kubernetes.io/projected/1b782f74-31e7-4fe9-809d-d0400a823e76-kube-api-access-76mxk\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631163    5906 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-console-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631174    5906 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631184    5906 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b782f74-31e7-4fe9-809d-d0400a823e76-tmp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631193    5906 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b782f74-31e7-4fe9-809d-d0400a823e76-console-oauth-config\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631202    5906 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-trusted-ca-bundle\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631234    5906 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:13.631453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:13.631243    5906 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b782f74-31e7-4fe9-809d-d0400a823e76-oauth-serving-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:32:14.206861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.206827    5906 generic.go:356] "Generic (PLEG): container finished" podID="1b782f74-31e7-4fe9-809d-d0400a823e76" containerID="3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de" exitCode=0
Oct 31 03:32:14.206861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.206871    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d9bcc8bff-lrgdt"
Oct 31 03:32:14.207082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.206897    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9bcc8bff-lrgdt" event={"ID":"1b782f74-31e7-4fe9-809d-d0400a823e76","Type":"ContainerDied","Data":"3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de"}
Oct 31 03:32:14.207082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.206942    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d9bcc8bff-lrgdt" event={"ID":"1b782f74-31e7-4fe9-809d-d0400a823e76","Type":"ContainerDied","Data":"d4f12df3b8ea5131f9e4684c9e6245aec85fdc17347e410f696dff2e41a787eb"}
Oct 31 03:32:14.207082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.206965    5906 scope.go:122] "RemoveContainer" containerID="3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de"
Oct 31 03:32:14.215674 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.215654    5906 scope.go:122] "RemoveContainer" containerID="3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de"
Oct 31 03:32:14.215961 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:32:14.215945    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de\": container with ID starting with 3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de not found: ID does not exist" containerID="3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de"
Oct 31 03:32:14.215999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.215970    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de"} err="failed to get container status \"3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de\": rpc error: code = NotFound desc = could not find container \"3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de\": container with ID starting with 3a33923459eb441aa6eba517ab1f2c47eee464b1c1382a849c19f2ab20f009de not found: ID does not exist"
Oct 31 03:32:14.376531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.376497    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d9bcc8bff-lrgdt"]
Oct 31 03:32:14.427903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.427870    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d9bcc8bff-lrgdt"]
Oct 31 03:32:14.832584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:14.832552    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:32:16.350135 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:32:16.350102    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b782f74-31e7-4fe9-809d-d0400a823e76" path="/var/lib/kubelet/pods/1b782f74-31e7-4fe9-809d-d0400a823e76/volumes"
Oct 31 03:32:36.380940 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:32:36.380906    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:33:06.378970 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:33:06.378934    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:33:15.202373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:33:15.202339    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:33:36.380115 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:33:36.380077    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:34:06.379762 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:34:06.379676    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:34:15.544906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:15.544871    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:17.386021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:17.385982    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:17.558660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:17.558624    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:17.732435 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:17.732357    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:18.934258 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:18.934199    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:19.084572 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:19.084546    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:20.320323 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:20.320280    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:29.134259 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:29.134226    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:29.272010 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:29.271971    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:29.588690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:29.588656    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:29.814604 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:29.814569    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:29.956857 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:29.956825    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:33.575627 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:33.575592    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:36.380509 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:34:36.380473    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:34:41.445465 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:41.445427    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:34:41.585659 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:34:41.585619    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:01.698872 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:01.698839    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:01.839339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:01.839300    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:04.644295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:04.644262    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:04.783591 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:04.783559    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:06.379546 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:35:06.379508    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:35:14.970283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:14.970245    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:15.796568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:15.796527    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:15.977012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:15.976983    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:19.403088 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:19.402995    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:19.542432 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:19.542400    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:19.986341 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:19.986302    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:20.165021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:20.164982    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:22.295850 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.295814    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"]
Oct 31 03:35:22.296471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.296382    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1b782f74-31e7-4fe9-809d-d0400a823e76" containerName="console"
Oct 31 03:35:22.296471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.296402    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b782f74-31e7-4fe9-809d-d0400a823e76" containerName="console"
Oct 31 03:35:22.296596 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.296496    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b782f74-31e7-4fe9-809d-d0400a823e76" containerName="console"
Oct 31 03:35:22.299499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.299475    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.376905 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.376865    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c542bc06-201c-408a-bea5-430bb16fee42-tmp\") pod \"jobset-operator-674cd59dc5-4s6ms\" (UID: \"c542bc06-201c-408a-bea5-430bb16fee42\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.377117 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.376938    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dklk\" (UniqueName: \"kubernetes.io/projected/c542bc06-201c-408a-bea5-430bb16fee42-kube-api-access-7dklk\") pod \"jobset-operator-674cd59dc5-4s6ms\" (UID: \"c542bc06-201c-408a-bea5-430bb16fee42\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.407761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.407730    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"]
Oct 31 03:35:22.415428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.415402    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 31 03:35:22.415594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.415464    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:35:22.443066 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.443036    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-mbzt9\""
Oct 31 03:35:22.478477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.478438    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c542bc06-201c-408a-bea5-430bb16fee42-tmp\") pod \"jobset-operator-674cd59dc5-4s6ms\" (UID: \"c542bc06-201c-408a-bea5-430bb16fee42\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.478685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.478511    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dklk\" (UniqueName: \"kubernetes.io/projected/c542bc06-201c-408a-bea5-430bb16fee42-kube-api-access-7dklk\") pod \"jobset-operator-674cd59dc5-4s6ms\" (UID: \"c542bc06-201c-408a-bea5-430bb16fee42\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.478940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.478914    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c542bc06-201c-408a-bea5-430bb16fee42-tmp\") pod \"jobset-operator-674cd59dc5-4s6ms\" (UID: \"c542bc06-201c-408a-bea5-430bb16fee42\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.542246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.542188    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dklk\" (UniqueName: \"kubernetes.io/projected/c542bc06-201c-408a-bea5-430bb16fee42-kube-api-access-7dklk\") pod \"jobset-operator-674cd59dc5-4s6ms\" (UID: \"c542bc06-201c-408a-bea5-430bb16fee42\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.625585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.625503    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"
Oct 31 03:35:22.767895 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.767864    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:22.919744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.919662    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:22.999385 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:22.999352    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms"]
Oct 31 03:35:23.013097 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:35:23.013070    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc542bc06_201c_408a_bea5_430bb16fee42.slice/crio-8b2dfb3a238716b8020515d451cafb00a845bd2bb861d2bba629342c1faaea70 WatchSource:0}: Error finding container 8b2dfb3a238716b8020515d451cafb00a845bd2bb861d2bba629342c1faaea70: Status 404 returned error can't find the container with id 8b2dfb3a238716b8020515d451cafb00a845bd2bb861d2bba629342c1faaea70
Oct 31 03:35:23.014579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:23.014558    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:35:23.103832 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:23.103802    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:23.909396 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:23.909359    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms" event={"ID":"c542bc06-201c-408a-bea5-430bb16fee42","Type":"ContainerStarted","Data":"8b2dfb3a238716b8020515d451cafb00a845bd2bb861d2bba629342c1faaea70"}
Oct 31 03:35:24.054684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.054654    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"]
Oct 31 03:35:24.059118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.059094    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.092452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.092417    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4tq4\" (UniqueName: \"kubernetes.io/projected/53245dfd-7920-4848-8a73-654d431404d5-kube-api-access-d4tq4\") pod \"cert-manager-webhook-5dbbbf75cc-mwwbv\" (UID: \"53245dfd-7920-4848-8a73-654d431404d5\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.092645 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.092468    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53245dfd-7920-4848-8a73-654d431404d5-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-mwwbv\" (UID: \"53245dfd-7920-4848-8a73-654d431404d5\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.119771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.119741    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"]
Oct 31 03:35:24.170232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.170112    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 31 03:35:24.170232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.170167    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 31 03:35:24.170429 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.170254    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-h4j8j\""
Oct 31 03:35:24.193504 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.193477    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4tq4\" (UniqueName: \"kubernetes.io/projected/53245dfd-7920-4848-8a73-654d431404d5-kube-api-access-d4tq4\") pod \"cert-manager-webhook-5dbbbf75cc-mwwbv\" (UID: \"53245dfd-7920-4848-8a73-654d431404d5\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.193672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.193520    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53245dfd-7920-4848-8a73-654d431404d5-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-mwwbv\" (UID: \"53245dfd-7920-4848-8a73-654d431404d5\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.256177 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.256143    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/53245dfd-7920-4848-8a73-654d431404d5-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-mwwbv\" (UID: \"53245dfd-7920-4848-8a73-654d431404d5\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.256350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.256233    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4tq4\" (UniqueName: \"kubernetes.io/projected/53245dfd-7920-4848-8a73-654d431404d5-kube-api-access-d4tq4\") pod \"cert-manager-webhook-5dbbbf75cc-mwwbv\" (UID: \"53245dfd-7920-4848-8a73-654d431404d5\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.340762 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.340726    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:24.371549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.371506    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:24.738928 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:35:24.738886    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53245dfd_7920_4848_8a73_654d431404d5.slice/crio-493eebc878f4a94e5bf6b18345340ad3442cdf921b8418369b4b0118d9a749d4 WatchSource:0}: Error finding container 493eebc878f4a94e5bf6b18345340ad3442cdf921b8418369b4b0118d9a749d4: Status 404 returned error can't find the container with id 493eebc878f4a94e5bf6b18345340ad3442cdf921b8418369b4b0118d9a749d4
Oct 31 03:35:24.740035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.740006    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"]
Oct 31 03:35:24.913763 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:24.913728    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv" event={"ID":"53245dfd-7920-4848-8a73-654d431404d5","Type":"ContainerStarted","Data":"493eebc878f4a94e5bf6b18345340ad3442cdf921b8418369b4b0118d9a749d4"}
Oct 31 03:35:25.814419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:25.814330    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:27.567847 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:27.567811    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:27.822613 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:27.822566    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:27.928231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:27.928174    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms" event={"ID":"c542bc06-201c-408a-bea5-430bb16fee42","Type":"ContainerStarted","Data":"8777f774354a50b27c9fa496ffdca80be5c7d081d2af90f971601c0ad69237dd"}
Oct 31 03:35:28.049432 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:28.049368    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-4s6ms" podStartSLOduration=1.692402323 podStartE2EDuration="6.049351269s" totalImagesPullingTime="4.356948946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:35:28.04842311 +0000 UTC m=+377.882283439" watchObservedRunningTime="2026-10-31 03:35:28.049351269 +0000 UTC m=+377.883211634"
Oct 31 03:35:28.692250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:28.692182    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:28.959912 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:28.959831    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"]
Oct 31 03:35:28.963854 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:28.963830    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.002638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.002596    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:29.039665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.039630    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/993ef660-46ae-49a4-ac80-e5a2c2751d58-manifests\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.039665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.039677    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mbsb\" (UniqueName: \"kubernetes.io/projected/993ef660-46ae-49a4-ac80-e5a2c2751d58-kube-api-access-4mbsb\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.039901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.039764    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/993ef660-46ae-49a4-ac80-e5a2c2751d58-tmp\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.039901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.039812    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.039901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.039866    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-metrics-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.040012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.039904    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.080121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.080088    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 03:35:29.080314 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.080182    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 03:35:29.080720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.080697    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 03:35:29.080720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.080706    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 03:35:29.080922 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.080725    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 03:35:29.080922 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.080829    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-lljsf\""
Oct 31 03:35:29.099458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.099429    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"]
Oct 31 03:35:29.141063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141018    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.141283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141109    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/993ef660-46ae-49a4-ac80-e5a2c2751d58-manifests\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.141283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141145    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mbsb\" (UniqueName: \"kubernetes.io/projected/993ef660-46ae-49a4-ac80-e5a2c2751d58-kube-api-access-4mbsb\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.141283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141180    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/993ef660-46ae-49a4-ac80-e5a2c2751d58-tmp\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.141283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141239    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.141520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141308    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-metrics-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.141620 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.141594    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/993ef660-46ae-49a4-ac80-e5a2c2751d58-manifests\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.143609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.143571    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/993ef660-46ae-49a4-ac80-e5a2c2751d58-tmp\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.143759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.143739    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.143802 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.143781    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.143850 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.143834    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/993ef660-46ae-49a4-ac80-e5a2c2751d58-metrics-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.151487 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.151456    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:29.209744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.209716    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mbsb\" (UniqueName: \"kubernetes.io/projected/993ef660-46ae-49a4-ac80-e5a2c2751d58-kube-api-access-4mbsb\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m\" (UID: \"993ef660-46ae-49a4-ac80-e5a2c2751d58\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.280866 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.280834    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:29.291104 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.291081    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:29.475447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.475410    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:29.616231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.616173    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:29.762736 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.762701    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"]
Oct 31 03:35:29.763093 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:35:29.762733    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993ef660_46ae_49a4_ac80_e5a2c2751d58.slice/crio-c2ea9a3da3543dc4755fecd0b7a97d351f7516c863fd9153843dccd511b82bff WatchSource:0}: Error finding container c2ea9a3da3543dc4755fecd0b7a97d351f7516c863fd9153843dccd511b82bff: Status 404 returned error can't find the container with id c2ea9a3da3543dc4755fecd0b7a97d351f7516c863fd9153843dccd511b82bff
Oct 31 03:35:29.763093 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.762926    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:29.937580 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.937499    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv" event={"ID":"53245dfd-7920-4848-8a73-654d431404d5","Type":"ContainerStarted","Data":"20c894900f6dabcecb4cbeda915ecfacdcf15831ed39571e844408fae2e2ebd5"}
Oct 31 03:35:29.937744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.937611    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:29.939058 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:29.939028    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m" event={"ID":"993ef660-46ae-49a4-ac80-e5a2c2751d58","Type":"ContainerStarted","Data":"c2ea9a3da3543dc4755fecd0b7a97d351f7516c863fd9153843dccd511b82bff"}
Oct 31 03:35:30.056744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:30.056691    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv" podStartSLOduration=1.608237517 podStartE2EDuration="6.056677075s" totalImagesPullingTime="4.448439558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:35:30.056166893 +0000 UTC m=+379.890027197" watchObservedRunningTime="2026-10-31 03:35:30.056677075 +0000 UTC m=+379.890537376"
Oct 31 03:35:32.425224 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:32.425158    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:33.750384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:33.750341    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:34.115513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:34.115484    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:34.263164 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:34.263132    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:34.966169 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:34.966081    5906 generic.go:356] "Generic (PLEG): container finished" podID="993ef660-46ae-49a4-ac80-e5a2c2751d58" containerID="3949aa78df8cdc1b798b8dd26fd3f5de5ac35a043cc53dadc2d9dc87111ba24b" exitCode=0
Oct 31 03:35:34.966169 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:34.966142    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m" event={"ID":"993ef660-46ae-49a4-ac80-e5a2c2751d58","Type":"ContainerDied","Data":"3949aa78df8cdc1b798b8dd26fd3f5de5ac35a043cc53dadc2d9dc87111ba24b"}
Oct 31 03:35:35.946458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:35.946427    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-mwwbv"
Oct 31 03:35:36.372467 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:36.372431    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:36.383929 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:35:36.383895    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:35:36.978281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:36.978240    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m" event={"ID":"993ef660-46ae-49a4-ac80-e5a2c2751d58","Type":"ContainerStarted","Data":"e742d31f3e9caaa727e5ed1becd9394659d537ad8bb6954d2902e698e0027fec"}
Oct 31 03:35:36.978459 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:36.978386    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:37.252886 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:37.252845    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:37.279723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:37.279670    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m" podStartSLOduration=3.487358757 podStartE2EDuration="9.27965602s" totalImagesPullingTime="5.236819967s" totalInitContainerRuntime="555.477296ms" isStatefulPod=false podCreationTimestamp="2026-10-31 03:35:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:35:37.227945853 +0000 UTC m=+387.061806155" watchObservedRunningTime="2026-10-31 03:35:37.27965602 +0000 UTC m=+387.113516324"
Oct 31 03:35:37.770545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:37.770512    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:38.444340 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:38.444305    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:40.152312 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:40.152277    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:40.292276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:40.292244    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:41.009833 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:41.009799    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:41.192636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:41.192594    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:42.368585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:42.368545    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:43.010470 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.010436    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:43.396287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.396190    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-bz66r"]
Oct 31 03:35:43.402769 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.402745    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.458464 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.458428    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-bz66r"]
Oct 31 03:35:43.476201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.476150    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51268a00-4d57-40cc-b2a3-d719316783be-bound-sa-token\") pod \"cert-manager-597b95cb84-bz66r\" (UID: \"51268a00-4d57-40cc-b2a3-d719316783be\") " pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.476401 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.476382    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2m6q\" (UniqueName: \"kubernetes.io/projected/51268a00-4d57-40cc-b2a3-d719316783be-kube-api-access-j2m6q\") pod \"cert-manager-597b95cb84-bz66r\" (UID: \"51268a00-4d57-40cc-b2a3-d719316783be\") " pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.507812 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.507781    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-j7szv\""
Oct 31 03:35:43.577714 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.577674    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2m6q\" (UniqueName: \"kubernetes.io/projected/51268a00-4d57-40cc-b2a3-d719316783be-kube-api-access-j2m6q\") pod \"cert-manager-597b95cb84-bz66r\" (UID: \"51268a00-4d57-40cc-b2a3-d719316783be\") " pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.577875 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.577775    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51268a00-4d57-40cc-b2a3-d719316783be-bound-sa-token\") pod \"cert-manager-597b95cb84-bz66r\" (UID: \"51268a00-4d57-40cc-b2a3-d719316783be\") " pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.642922 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.642889    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51268a00-4d57-40cc-b2a3-d719316783be-bound-sa-token\") pod \"cert-manager-597b95cb84-bz66r\" (UID: \"51268a00-4d57-40cc-b2a3-d719316783be\") " pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.643115 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.643091    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2m6q\" (UniqueName: \"kubernetes.io/projected/51268a00-4d57-40cc-b2a3-d719316783be-kube-api-access-j2m6q\") pod \"cert-manager-597b95cb84-bz66r\" (UID: \"51268a00-4d57-40cc-b2a3-d719316783be\") " pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:43.714851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:43.714751    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-bz66r"
Oct 31 03:35:44.086033 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:35:44.086004    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51268a00_4d57_40cc_b2a3_d719316783be.slice/crio-3c42da6f38eb39e39eb5b3eab83c2890705c5b02a28d2b8f781bb98e32712054 WatchSource:0}: Error finding container 3c42da6f38eb39e39eb5b3eab83c2890705c5b02a28d2b8f781bb98e32712054: Status 404 returned error can't find the container with id 3c42da6f38eb39e39eb5b3eab83c2890705c5b02a28d2b8f781bb98e32712054
Oct 31 03:35:44.087223 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:44.087181    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-bz66r"]
Oct 31 03:35:45.012126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:45.012087    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-bz66r" event={"ID":"51268a00-4d57-40cc-b2a3-d719316783be","Type":"ContainerStarted","Data":"a429b772c4c67246df9d8c2c763439bad96cdd3398fa6ea61b9522e0e4069246"}
Oct 31 03:35:45.012556 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:45.012137    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-bz66r" event={"ID":"51268a00-4d57-40cc-b2a3-d719316783be","Type":"ContainerStarted","Data":"3c42da6f38eb39e39eb5b3eab83c2890705c5b02a28d2b8f781bb98e32712054"}
Oct 31 03:35:45.133256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:45.133172    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-bz66r" podStartSLOduration=2.133157715 podStartE2EDuration="2.133157715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:35:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:35:45.132701905 +0000 UTC m=+394.966562210" watchObservedRunningTime="2026-10-31 03:35:45.133157715 +0000 UTC m=+394.967018016"
Oct 31 03:35:45.603664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:45.603619    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:45.790054 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:45.789983    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:47.427962 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:47.427916    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:47.985340 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:47.985310    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-69m4m"
Oct 31 03:35:48.977853 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:48.977820    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:49.129423 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:49.129390    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:53.204404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.204369    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:53.220718 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.220683    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"]
Oct 31 03:35:53.265592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.265559    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.278990 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.278962    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"]
Oct 31 03:35:53.329563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.329531    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 31 03:35:53.329563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.329551    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kk2nt\""
Oct 31 03:35:53.354689 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.354658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:53.365824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.365787    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/410a60bd-b93a-458e-a814-3b3a01d4b074-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.365972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.365845    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzpp7\" (UniqueName: \"kubernetes.io/projected/410a60bd-b93a-458e-a814-3b3a01d4b074-kube-api-access-bzpp7\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.365972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.365933    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/410a60bd-b93a-458e-a814-3b3a01d4b074-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.371304 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.371282    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 31 03:35:53.371419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.371305    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 31 03:35:53.371419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.371283    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 31 03:35:53.467397 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.467313    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzpp7\" (UniqueName: \"kubernetes.io/projected/410a60bd-b93a-458e-a814-3b3a01d4b074-kube-api-access-bzpp7\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.467397 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.467353    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/410a60bd-b93a-458e-a814-3b3a01d4b074-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.467585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.467428    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/410a60bd-b93a-458e-a814-3b3a01d4b074-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.468061 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.468037    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/410a60bd-b93a-458e-a814-3b3a01d4b074-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.469900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.469881    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/410a60bd-b93a-458e-a814-3b3a01d4b074-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.532856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.532819    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzpp7\" (UniqueName: \"kubernetes.io/projected/410a60bd-b93a-458e-a814-3b3a01d4b074-kube-api-access-bzpp7\") pod \"kuadrant-console-plugin-66c4fb575d-2hbr7\" (UID: \"410a60bd-b93a-458e-a814-3b3a01d4b074\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.595276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.595232    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"
Oct 31 03:35:53.662559 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.661920    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:53.808219 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.808158    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:53.951644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.951608    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:35:53.981241 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:35:53.981193    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod410a60bd_b93a_458e_a814_3b3a01d4b074.slice/crio-2dfe6138f57d577d037fdeeb1c1ff7f9b876f03db84d467c8f7f88a7f856421f WatchSource:0}: Error finding container 2dfe6138f57d577d037fdeeb1c1ff7f9b876f03db84d467c8f7f88a7f856421f: Status 404 returned error can't find the container with id 2dfe6138f57d577d037fdeeb1c1ff7f9b876f03db84d467c8f7f88a7f856421f
Oct 31 03:35:53.982499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:53.982455    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7"]
Oct 31 03:35:54.051560 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:35:54.051526    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7" event={"ID":"410a60bd-b93a-458e-a814-3b3a01d4b074","Type":"ContainerStarted","Data":"2dfe6138f57d577d037fdeeb1c1ff7f9b876f03db84d467c8f7f88a7f856421f"}
Oct 31 03:36:00.084939 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:00.084899    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7" event={"ID":"410a60bd-b93a-458e-a814-3b3a01d4b074","Type":"ContainerStarted","Data":"56042f4584dc492fa0641bab3f83d8f233b01a56f7749ebcbeebb86b628e9979"}
Oct 31 03:36:00.248967 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:00.248913    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-2hbr7" podStartSLOduration=1.627315705 podStartE2EDuration="7.248899255s" totalImagesPullingTime="5.62158355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:00.248284134 +0000 UTC m=+410.082144439" watchObservedRunningTime="2026-10-31 03:36:00.248899255 +0000 UTC m=+410.082759557"
Oct 31 03:36:01.069118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:01.069085    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:01.250303 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:01.250267    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:01.724773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:01.724735    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:02.454269 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:02.454232    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:02.607035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:02.606994    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:03.164350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.164315    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:03.192787 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.192751    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5897747699-5zggr"]
Oct 31 03:36:03.196403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.196385    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.259375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.259332    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7b42a40-d70d-4b02-8bac-1056be986ecc-tls-certs\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.259547 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.259382    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vflbc\" (UniqueName: \"kubernetes.io/projected/c7b42a40-d70d-4b02-8bac-1056be986ecc-kube-api-access-vflbc\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.259547 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.259458    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7b42a40-d70d-4b02-8bac-1056be986ecc-tmp\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.290128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.290089    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mclvc\""
Oct 31 03:36:03.290128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.290089    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 31 03:36:03.302109 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.302078    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 03:36:03.302109 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.302078    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 31 03:36:03.302339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.302140    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 03:36:03.303201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.303181    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5897747699-5zggr"]
Oct 31 03:36:03.310887 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.310863    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:03.360831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.360791    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7b42a40-d70d-4b02-8bac-1056be986ecc-tls-certs\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.361036 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.360848    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vflbc\" (UniqueName: \"kubernetes.io/projected/c7b42a40-d70d-4b02-8bac-1056be986ecc-kube-api-access-vflbc\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.361036 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.360898    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7b42a40-d70d-4b02-8bac-1056be986ecc-tmp\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.363134 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.363108    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7b42a40-d70d-4b02-8bac-1056be986ecc-tmp\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.363488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.363464    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7b42a40-d70d-4b02-8bac-1056be986ecc-tls-certs\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.423582 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.423507    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vflbc\" (UniqueName: \"kubernetes.io/projected/c7b42a40-d70d-4b02-8bac-1056be986ecc-kube-api-access-vflbc\") pod \"kube-auth-proxy-5897747699-5zggr\" (UID: \"c7b42a40-d70d-4b02-8bac-1056be986ecc\") " pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.508291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.508237    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr"
Oct 31 03:36:03.659466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.659429    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:03.807914 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.807876    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:03.882799 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:36:03.882765    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b42a40_d70d_4b02_8bac_1056be986ecc.slice/crio-be1cc4caf98c0c325a4052b1de3e7a874ca58ffa3299ecd65bb924d86c683996 WatchSource:0}: Error finding container be1cc4caf98c0c325a4052b1de3e7a874ca58ffa3299ecd65bb924d86c683996: Status 404 returned error can't find the container with id be1cc4caf98c0c325a4052b1de3e7a874ca58ffa3299ecd65bb924d86c683996
Oct 31 03:36:03.884025 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.884005    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5897747699-5zggr"]
Oct 31 03:36:03.974755 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:03.974723    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:04.106748 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:04.106664    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr" event={"ID":"c7b42a40-d70d-4b02-8bac-1056be986ecc","Type":"ContainerStarted","Data":"be1cc4caf98c0c325a4052b1de3e7a874ca58ffa3299ecd65bb924d86c683996"}
Oct 31 03:36:05.441836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:05.441771    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:05.580365 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:05.580330    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:06.384184 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:36:06.384144    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:36:08.081228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.081174    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:08.124856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.124818    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr" event={"ID":"c7b42a40-d70d-4b02-8bac-1056be986ecc","Type":"ContainerStarted","Data":"5d1087156dfd02983fdd08394f77853453e9a1f96099a8abc8fcbfb1293d99ab"}
Oct 31 03:36:08.226812 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.226778    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:08.246583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.246526    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5897747699-5zggr" podStartSLOduration=1.848095084 podStartE2EDuration="5.246514467s" totalImagesPullingTime="3.398419383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:08.246352112 +0000 UTC m=+418.080212431" watchObservedRunningTime="2026-10-31 03:36:08.246514467 +0000 UTC m=+418.080374769"
Oct 31 03:36:08.482548 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.482469    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:08.571876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.571841    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:08.575189 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.575172    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.609964 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.609918    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file\") pod \"limitador-limitador-74dfcfb894-r87g8\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.610150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.610041    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwvt6\" (UniqueName: \"kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6\") pod \"limitador-limitador-74dfcfb894-r87g8\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.629993 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.629964    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:08.710750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.710718    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwvt6\" (UniqueName: \"kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6\") pod \"limitador-limitador-74dfcfb894-r87g8\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.710905 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.710825    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file\") pod \"limitador-limitador-74dfcfb894-r87g8\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.716991 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.716974    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 31 03:36:08.721540 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.721521    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file\") pod \"limitador-limitador-74dfcfb894-r87g8\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.737035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.736909    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:08.773465 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.773432    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwvt6\" (UniqueName: \"kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6\") pod \"limitador-limitador-74dfcfb894-r87g8\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.812041 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.812007    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:08.887214 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.887179    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:08.889566 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:08.889535    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:09.256823 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:36:09.256791    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb3ea3df_4c42_4861_afec_686e24ff6276.slice/crio-e5249c5f699f697ade33fd713685e6c7f904a10b9a11db65fc33771485b4f0d2 WatchSource:0}: Error finding container e5249c5f699f697ade33fd713685e6c7f904a10b9a11db65fc33771485b4f0d2: Status 404 returned error can't find the container with id e5249c5f699f697ade33fd713685e6c7f904a10b9a11db65fc33771485b4f0d2
Oct 31 03:36:09.258435 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:09.258406    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:09.963794 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:09.963764    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:10.104041 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:10.104008    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:10.135338 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:10.135303    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8" event={"ID":"eb3ea3df-4c42-4861-afec-686e24ff6276","Type":"ContainerStarted","Data":"e5249c5f699f697ade33fd713685e6c7f904a10b9a11db65fc33771485b4f0d2"}
Oct 31 03:36:10.598520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:10.598490    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:12.458499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.458457    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-fpwkf"]
Oct 31 03:36:12.463863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.463828    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:36:12.516059 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.516024    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-fpwkf"]
Oct 31 03:36:12.546255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.546221    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl2w9\" (UniqueName: \"kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9\") pod \"authorino-56c6c9ddd5-fpwkf\" (UID: \"26e4d787-8c2a-4694-9efa-e96081870ea4\") " pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:36:12.566155 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.566127    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ppjlh\""
Oct 31 03:36:12.647328 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.647288    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl2w9\" (UniqueName: \"kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9\") pod \"authorino-56c6c9ddd5-fpwkf\" (UID: \"26e4d787-8c2a-4694-9efa-e96081870ea4\") " pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:36:12.710496 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.710432    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl2w9\" (UniqueName: \"kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9\") pod \"authorino-56c6c9ddd5-fpwkf\" (UID: \"26e4d787-8c2a-4694-9efa-e96081870ea4\") " pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:36:12.778189 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:12.778158    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:36:13.159079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:13.159036    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-fpwkf"]
Oct 31 03:36:13.448294 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:13.448258    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:13.594848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:13.594812    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:13.837400 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:13.837365    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:13.976190 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:13.976157    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:14.156266 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:14.156150    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8" event={"ID":"eb3ea3df-4c42-4861-afec-686e24ff6276","Type":"ContainerStarted","Data":"5a366ebee5bd08ab20cf20659e9ceac0ab2292cf4dde2ae0aa3ee4adb61ad2ac"}
Oct 31 03:36:14.156447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:14.156355    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:14.157363 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:14.157335    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf" event={"ID":"26e4d787-8c2a-4694-9efa-e96081870ea4","Type":"ContainerStarted","Data":"ed2eef8cbaaac885ba882d4f03ee8af21cee315b910ac8dec2b5d9ed685bc3ad"}
Oct 31 03:36:14.246998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:14.246968    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:14.275880 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:14.275823    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8" podStartSLOduration=2.169578144 podStartE2EDuration="6.275807032s" totalImagesPullingTime="4.106228888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:14.274861173 +0000 UTC m=+424.108721479" watchObservedRunningTime="2026-10-31 03:36:14.275807032 +0000 UTC m=+424.109667459"
Oct 31 03:36:14.389479 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:14.389449    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:15.563706 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:15.563673    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:15.717703 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:15.717672    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:15.868526 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:15.868436    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:16.213858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:16.213766    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:18.179934 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:18.179889    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf" event={"ID":"26e4d787-8c2a-4694-9efa-e96081870ea4","Type":"ContainerStarted","Data":"0d7014f21fac18bffe8ee6dda0f2fc18ccee35ff0be90466728fee12e715c9bc"}
Oct 31 03:36:18.297600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:18.297543    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf" podStartSLOduration=2.366175259 podStartE2EDuration="6.29753081s" totalImagesPullingTime="3.931355551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:18.297287136 +0000 UTC m=+428.131147442" watchObservedRunningTime="2026-10-31 03:36:18.29753081 +0000 UTC m=+428.131391174"
Oct 31 03:36:22.258366 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:22.258327    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:22.399623 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:22.399593    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:25.164195 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:25.164165    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:25.250303 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:25.250268    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:25.392037 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:25.392007    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:26.728513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:26.728479    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:26.729004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:26.728703    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8" podUID="eb3ea3df-4c42-4861-afec-686e24ff6276" containerName="limitador" containerID="cri-o://5a366ebee5bd08ab20cf20659e9ceac0ab2292cf4dde2ae0aa3ee4adb61ad2ac" gracePeriod=30
Oct 31 03:36:27.218705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.218671    5906 generic.go:356] "Generic (PLEG): container finished" podID="eb3ea3df-4c42-4861-afec-686e24ff6276" containerID="5a366ebee5bd08ab20cf20659e9ceac0ab2292cf4dde2ae0aa3ee4adb61ad2ac" exitCode=0
Oct 31 03:36:27.218873 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.218711    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8" event={"ID":"eb3ea3df-4c42-4861-afec-686e24ff6276","Type":"ContainerDied","Data":"5a366ebee5bd08ab20cf20659e9ceac0ab2292cf4dde2ae0aa3ee4adb61ad2ac"}
Oct 31 03:36:27.673893 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.673859    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:27.789674 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.789637    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file\" (UniqueName: \"kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file\") pod \"eb3ea3df-4c42-4861-afec-686e24ff6276\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") "
Oct 31 03:36:27.790172 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.789700    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6\" (UniqueName: \"kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6\") pod \"eb3ea3df-4c42-4861-afec-686e24ff6276\" (UID: \"eb3ea3df-4c42-4861-afec-686e24ff6276\") "
Oct 31 03:36:27.790172 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.790103    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file" pod "eb3ea3df-4c42-4861-afec-686e24ff6276" (UID: "eb3ea3df-4c42-4861-afec-686e24ff6276"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:36:27.792169 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.792108    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6" pod "eb3ea3df-4c42-4861-afec-686e24ff6276" (UID: "eb3ea3df-4c42-4861-afec-686e24ff6276"). InnerVolumeSpecName "kube-api-access-lwvt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:36:27.890669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.890629    5906 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eb3ea3df-4c42-4861-afec-686e24ff6276-config-file\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:36:27.890669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:27.890661    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwvt6\" (UniqueName: \"kubernetes.io/projected/eb3ea3df-4c42-4861-afec-686e24ff6276-kube-api-access-lwvt6\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:36:28.034997 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.034963    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"]
Oct 31 03:36:28.035350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.035336    5906 cpu_manager.go:414] "containerMap: removing container" podUID="eb3ea3df-4c42-4861-afec-686e24ff6276" containerName="limitador"
Oct 31 03:36:28.035404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.035351    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb3ea3df-4c42-4861-afec-686e24ff6276" containerName="limitador"
Oct 31 03:36:28.035452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.035443    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb3ea3df-4c42-4861-afec-686e24ff6276" containerName="limitador"
Oct 31 03:36:28.038599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.038583    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.095653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.095575    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 03:36:28.095653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.095590    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 03:36:28.095653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.095661    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-t4hnj\""
Oct 31 03:36:28.193192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.193155    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95ce020-c6a0-415f-af9c-52d50352f106-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.193393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.193287    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e95ce020-c6a0-415f-af9c-52d50352f106-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.193393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.193335    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cn6\" (UniqueName: \"kubernetes.io/projected/e95ce020-c6a0-415f-af9c-52d50352f106-kube-api-access-t4cn6\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.193393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.193371    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e95ce020-c6a0-415f-af9c-52d50352f106-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.199526 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.199501    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 31 03:36:28.200777 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.200762    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"]
Oct 31 03:36:28.224107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.224077    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8"
Oct 31 03:36:28.224338 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.224076    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-r87g8" event={"ID":"eb3ea3df-4c42-4861-afec-686e24ff6276","Type":"ContainerDied","Data":"e5249c5f699f697ade33fd713685e6c7f904a10b9a11db65fc33771485b4f0d2"}
Oct 31 03:36:28.224338 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.224261    5906 scope.go:122] "RemoveContainer" containerID="5a366ebee5bd08ab20cf20659e9ceac0ab2292cf4dde2ae0aa3ee4adb61ad2ac"
Oct 31 03:36:28.294640 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.294605    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cn6\" (UniqueName: \"kubernetes.io/projected/e95ce020-c6a0-415f-af9c-52d50352f106-kube-api-access-t4cn6\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.294851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.294671    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e95ce020-c6a0-415f-af9c-52d50352f106-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.294851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.294747    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95ce020-c6a0-415f-af9c-52d50352f106-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.294851 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.294816    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e95ce020-c6a0-415f-af9c-52d50352f106-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.295132 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.295080    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e95ce020-c6a0-415f-af9c-52d50352f106-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.295225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.295175    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e95ce020-c6a0-415f-af9c-52d50352f106-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.295574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.295554    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e95ce020-c6a0-415f-af9c-52d50352f106-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.357543 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.357511    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cn6\" (UniqueName: \"kubernetes.io/projected/e95ce020-c6a0-415f-af9c-52d50352f106-kube-api-access-t4cn6\") pod \"kserve-module-controller-manager-7566cf5fc8-4cxnd\" (UID: \"e95ce020-c6a0-415f-af9c-52d50352f106\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:28.393170 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.393129    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:28.397094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.397066    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-r87g8"]
Oct 31 03:36:28.651460 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:28.651367    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:29.071289 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:36:29.071251    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode95ce020_c6a0_415f_af9c_52d50352f106.slice/crio-e8ae7536d2affc859228960282dd375838e7a2837d61437efc0fc5a339d422fe WatchSource:0}: Error finding container e8ae7536d2affc859228960282dd375838e7a2837d61437efc0fc5a339d422fe: Status 404 returned error can't find the container with id e8ae7536d2affc859228960282dd375838e7a2837d61437efc0fc5a339d422fe
Oct 31 03:36:29.072730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:29.072704    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"]
Oct 31 03:36:29.229730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:29.229693    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd" event={"ID":"e95ce020-c6a0-415f-af9c-52d50352f106","Type":"ContainerStarted","Data":"e8ae7536d2affc859228960282dd375838e7a2837d61437efc0fc5a339d422fe"}
Oct 31 03:36:30.351549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:30.351516    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb3ea3df-4c42-4861-afec-686e24ff6276" path="/var/lib/kubelet/pods/eb3ea3df-4c42-4861-afec-686e24ff6276/volumes"
Oct 31 03:36:32.246568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:32.246531    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd" event={"ID":"e95ce020-c6a0-415f-af9c-52d50352f106","Type":"ContainerStarted","Data":"67c6aa0bb3698d3a658e1ae82f907566daeca475115e56f6a86b08a9eba5ca7e"}
Oct 31 03:36:32.247035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:32.246647    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:32.420954 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:32.420901    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd" podStartSLOduration=2.683895886 podStartE2EDuration="5.420886609s" totalImagesPullingTime="2.736990723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:32.420275951 +0000 UTC m=+442.254136262" watchObservedRunningTime="2026-10-31 03:36:32.420886609 +0000 UTC m=+442.254746911"
Oct 31 03:36:33.660568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:33.660535    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:33.807783 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:33.807744    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:34.393789 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.393748    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"]
Oct 31 03:36:34.397376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.397350    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:34.453233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.453181    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"]
Oct 31 03:36:34.479826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.479793    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:34.503522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.503488    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-tlkqs\""
Oct 31 03:36:34.503522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.503491    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 03:36:34.503522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.503491    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 31 03:36:34.503522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.503499    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 03:36:34.557994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.557959    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/948ce7a9-1e54-40aa-b7af-8d8a90567780-data\") pod \"postgres-7cdbf6dc44-5qnvm\" (UID: \"948ce7a9-1e54-40aa-b7af-8d8a90567780\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:34.557994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.557999    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvrtd\" (UniqueName: \"kubernetes.io/projected/948ce7a9-1e54-40aa-b7af-8d8a90567780-kube-api-access-wvrtd\") pod \"postgres-7cdbf6dc44-5qnvm\" (UID: \"948ce7a9-1e54-40aa-b7af-8d8a90567780\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:34.658693 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.658598    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/948ce7a9-1e54-40aa-b7af-8d8a90567780-data\") pod \"postgres-7cdbf6dc44-5qnvm\" (UID: \"948ce7a9-1e54-40aa-b7af-8d8a90567780\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:34.658693 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.658640    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvrtd\" (UniqueName: \"kubernetes.io/projected/948ce7a9-1e54-40aa-b7af-8d8a90567780-kube-api-access-wvrtd\") pod \"postgres-7cdbf6dc44-5qnvm\" (UID: \"948ce7a9-1e54-40aa-b7af-8d8a90567780\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:34.659045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.659022    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/948ce7a9-1e54-40aa-b7af-8d8a90567780-data\") pod \"postgres-7cdbf6dc44-5qnvm\" (UID: \"948ce7a9-1e54-40aa-b7af-8d8a90567780\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:34.721641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:34.721603    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvrtd\" (UniqueName: \"kubernetes.io/projected/948ce7a9-1e54-40aa-b7af-8d8a90567780-kube-api-access-wvrtd\") pod \"postgres-7cdbf6dc44-5qnvm\" (UID: \"948ce7a9-1e54-40aa-b7af-8d8a90567780\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:35.010137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:35.010095    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:35.383036 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:36:35.383005    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948ce7a9_1e54_40aa_b7af_8d8a90567780.slice/crio-dd43e59550d60c8d4b1f88a922368330d0977f6669cf90a4bf0ba48f363cd3f4 WatchSource:0}: Error finding container dd43e59550d60c8d4b1f88a922368330d0977f6669cf90a4bf0ba48f363cd3f4: Status 404 returned error can't find the container with id dd43e59550d60c8d4b1f88a922368330d0977f6669cf90a4bf0ba48f363cd3f4
Oct 31 03:36:35.383953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:35.383928    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"]
Oct 31 03:36:36.265622 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:36.265586    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm" event={"ID":"948ce7a9-1e54-40aa-b7af-8d8a90567780","Type":"ContainerStarted","Data":"dd43e59550d60c8d4b1f88a922368330d0977f6669cf90a4bf0ba48f363cd3f4"}
Oct 31 03:36:36.382340 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:36:36.382303    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:36:37.396195 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:37.396141    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:37.537223 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:37.537182    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:43.300805 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:43.300769    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm" event={"ID":"948ce7a9-1e54-40aa-b7af-8d8a90567780","Type":"ContainerStarted","Data":"1c5a4f4941788bb50efc62fe1014a3f7608e0d2b0678457d8a3e74019215c972"}
Oct 31 03:36:43.301256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:43.300891    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:43.420229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:43.420153    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm" podStartSLOduration=2.5105515819999997 podStartE2EDuration="9.420140202s" totalImagesPullingTime="6.90958862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:43.419723038 +0000 UTC m=+453.253583352" watchObservedRunningTime="2026-10-31 03:36:43.420140202 +0000 UTC m=+453.254000507"
Oct 31 03:36:49.334861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:49.334787    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-5qnvm"
Oct 31 03:36:50.733092 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:50.733057    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:50.870406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:50.870371    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:36:53.251907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:53.251876    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-4cxnd"
Oct 31 03:36:59.859291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:36:59.859248    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:00.003257 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:00.003223    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:03.517386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:03.517352    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:06.382347 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:37:06.382311    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:37:06.741660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:06.741630    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:06.881969 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:06.881936    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:07.022567 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:07.022445    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:07.159738 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:07.159703    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:10.652150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:10.652060    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:11.453131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:11.453089    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:11.597273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:11.597233    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:12.563630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:12.563592    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:14.582888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:14.582845    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:14.759669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:14.759631    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:16.548193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:16.548140    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:30.026594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.026560    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"]
Oct 31 03:37:30.033627 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.033601    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088478 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088441    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 03:37:30.088673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088574    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088586    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 03:37:30.088673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088604    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 03:37:30.088673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088608    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bfz2c\""
Oct 31 03:37:30.088673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088629    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 03:37:30.088673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088623    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088899 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088736    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088899 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088766    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088899 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088785    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088900    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.088998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.088952    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l245l\" (UniqueName: \"kubernetes.io/projected/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-kube-api-access-l245l\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.141620 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.141580    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"]
Oct 31 03:37:30.189566 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189534    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.189751 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189574    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.189751 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189592    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.189751 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189619    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.189751 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189645    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l245l\" (UniqueName: \"kubernetes.io/projected/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-kube-api-access-l245l\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.189751 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189675    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.189751 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.189706    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.190490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.190463    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.192022 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.191991    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.192146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.192066    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.192351 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.192331    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.192433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.192406    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.252893 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.252855    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.253068 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.253045    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l245l\" (UniqueName: \"kubernetes.io/projected/e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc-kube-api-access-l245l\") pod \"istiod-openshift-gateway-7ddd9cfdc8-qns5d\" (UID: \"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.345311 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.345197    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:30.766517 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:37:30.766487    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode72c8692_ad29_4343_a3dd_5f5ca3cc4fdc.slice/crio-c26d99745797326b6e75fd9986144b9befe8d5049a9ca5c5c04c00bcb8620972 WatchSource:0}: Error finding container c26d99745797326b6e75fd9986144b9befe8d5049a9ca5c5c04c00bcb8620972: Status 404 returned error can't find the container with id c26d99745797326b6e75fd9986144b9befe8d5049a9ca5c5c04c00bcb8620972
Oct 31 03:37:30.766689 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:30.766654    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"]
Oct 31 03:37:31.510615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:31.510577    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d" event={"ID":"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc","Type":"ContainerStarted","Data":"c26d99745797326b6e75fd9986144b9befe8d5049a9ca5c5c04c00bcb8620972"}
Oct 31 03:37:34.568451 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:34.568415    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:34.717161 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:34.717120    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:34.857550 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:34.857461    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:35.002836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.002802    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:35.156778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.156628    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:35.296809 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.296779    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:35.312999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.312957    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:37:35.313258 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.313025    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:37:35.442589 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.442513    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:35.530654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.530617    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d" event={"ID":"e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc","Type":"ContainerStarted","Data":"082c228b8cd535cc13e6f1a948d0e2cd85ccf1d7629b5d25f00dbfc07f651a45"}
Oct 31 03:37:35.530654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.530771    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:35.582312 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.582274    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:35.655457 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:35.655404    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d" podStartSLOduration=2.111776182 podStartE2EDuration="6.655384422s" totalImagesPullingTime="4.54360824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:37:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:37:35.654060017 +0000 UTC m=+505.487920322" watchObservedRunningTime="2026-10-31 03:37:35.655384422 +0000 UTC m=+505.489244727"
Oct 31 03:37:36.381475 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:37:36.381437    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:37:36.536765 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:36.536729    5906 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-qns5d container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 03:37:36.536928 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:36.536781    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d" podUID="e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 03:37:36.714870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:36.714790    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:39.536760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:39.536731    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-qns5d"
Oct 31 03:37:41.659718 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:41.659684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:41.840771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:41.840735    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:41.981098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:41.981014    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:42.121598 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:42.121561    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:42.315407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:42.315376    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:42.468085 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:42.468050    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:42.621244 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:42.621145    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:42.769854 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:42.769816    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:42.909730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:42.909622    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:44.488840 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:44.488808    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:44.642962 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:44.642928    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:44.783389 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:44.783339    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:44.985557 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:44.985522    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:53.419887 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:53.419848    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:56.264784 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:56.264750    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:37:56.549960 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:37:56.549878    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:38:06.380667 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:38:06.380564    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:38:10.429184 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.429145    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-fpwkf"]
Oct 31 03:38:10.429628 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.429394    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf" podUID="26e4d787-8c2a-4694-9efa-e96081870ea4" containerName="authorino" containerID="cri-o://0d7014f21fac18bffe8ee6dda0f2fc18ccee35ff0be90466728fee12e715c9bc" gracePeriod=30
Oct 31 03:38:10.683002 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.682928    5906 generic.go:356] "Generic (PLEG): container finished" podID="26e4d787-8c2a-4694-9efa-e96081870ea4" containerID="0d7014f21fac18bffe8ee6dda0f2fc18ccee35ff0be90466728fee12e715c9bc" exitCode=0
Oct 31 03:38:10.683139 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.683007    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf" event={"ID":"26e4d787-8c2a-4694-9efa-e96081870ea4","Type":"ContainerDied","Data":"0d7014f21fac18bffe8ee6dda0f2fc18ccee35ff0be90466728fee12e715c9bc"}
Oct 31 03:38:10.683139 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.683048    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf" event={"ID":"26e4d787-8c2a-4694-9efa-e96081870ea4","Type":"ContainerDied","Data":"ed2eef8cbaaac885ba882d4f03ee8af21cee315b910ac8dec2b5d9ed685bc3ad"}
Oct 31 03:38:10.683139 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.683059    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed2eef8cbaaac885ba882d4f03ee8af21cee315b910ac8dec2b5d9ed685bc3ad"
Oct 31 03:38:10.683139 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.683060    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:38:10.780497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.780458    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9\" (UniqueName: \"kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9\") pod \"26e4d787-8c2a-4694-9efa-e96081870ea4\" (UID: \"26e4d787-8c2a-4694-9efa-e96081870ea4\") "
Oct 31 03:38:10.782576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.782550    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9" pod "26e4d787-8c2a-4694-9efa-e96081870ea4" (UID: "26e4d787-8c2a-4694-9efa-e96081870ea4"). InnerVolumeSpecName "kube-api-access-jl2w9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:38:10.881678 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:10.881644    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jl2w9\" (UniqueName: \"kubernetes.io/projected/26e4d787-8c2a-4694-9efa-e96081870ea4-kube-api-access-jl2w9\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:38:11.687613 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:11.687582    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-fpwkf"
Oct 31 03:38:11.855465 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:11.855417    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-fpwkf"]
Oct 31 03:38:11.859260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:11.859234    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-fpwkf"]
Oct 31 03:38:12.351561 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:12.351523    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26e4d787-8c2a-4694-9efa-e96081870ea4" path="/var/lib/kubelet/pods/26e4d787-8c2a-4694-9efa-e96081870ea4/volumes"
Oct 31 03:38:16.926719 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:16.926683    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:38:36.380686 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:38:36.380647    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:38:47.566197 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:47.566146    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:38:47.707350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:47.707321    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:38:50.751232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:50.751172    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:38:50.891878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:38:50.891827    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:06.382100 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:39:06.382060    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:39:15.973276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:15.973240    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:16.123270 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:16.123236    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:17.220428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:17.220376    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:17.361361 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:17.361321    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:36.380974 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:39:36.380940    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:39:37.826727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:37.826693    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:41.712874 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:41.712838    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:46.097000 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.096969    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"]
Oct 31 03:39:46.097503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.097484    5906 cpu_manager.go:414] "containerMap: removing container" podUID="26e4d787-8c2a-4694-9efa-e96081870ea4" containerName="authorino"
Oct 31 03:39:46.097560 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.097505    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26e4d787-8c2a-4694-9efa-e96081870ea4" containerName="authorino"
Oct 31 03:39:46.097600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.097580    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="26e4d787-8c2a-4694-9efa-e96081870ea4" containerName="authorino"
Oct 31 03:39:46.100843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.100822    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.157484 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.157449    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"]
Oct 31 03:39:46.189633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.189595    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:46.200476 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.200447    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxqjk\" (UniqueName: \"kubernetes.io/projected/86ae1d89-4330-42ac-ba91-977fdb19878e-kube-api-access-jxqjk\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.200624 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.200504    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ae1d89-4330-42ac-ba91-977fdb19878e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.200624 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.200606    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.200740 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.200676    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.200802 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.200788    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.200865 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.200822    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.207493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.207471    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5hbl9\""
Oct 31 03:39:46.207493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.207471    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 31 03:39:46.207655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.207501    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 03:39:46.207655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.207530    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 03:39:46.302063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302029    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302073    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxqjk\" (UniqueName: \"kubernetes.io/projected/86ae1d89-4330-42ac-ba91-977fdb19878e-kube-api-access-jxqjk\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302111    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ae1d89-4330-42ac-ba91-977fdb19878e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302131    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302162    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302232    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302637 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302618    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302645    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.302695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.302618    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.304470 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.304448    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86ae1d89-4330-42ac-ba91-977fdb19878e-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.304645 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.304630    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86ae1d89-4330-42ac-ba91-977fdb19878e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.365100 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.365029    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxqjk\" (UniqueName: \"kubernetes.io/projected/86ae1d89-4330-42ac-ba91-977fdb19878e-kube-api-access-jxqjk\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc\" (UID: \"86ae1d89-4330-42ac-ba91-977fdb19878e\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.413711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.413674    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:46.780889 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:39:46.780860    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ae1d89_4330_42ac_ba91_977fdb19878e.slice/crio-506ee0ca05a26ee800752a90804431f7e4e646fa656ff81d89175b75bf50f959 WatchSource:0}: Error finding container 506ee0ca05a26ee800752a90804431f7e4e646fa656ff81d89175b75bf50f959: Status 404 returned error can't find the container with id 506ee0ca05a26ee800752a90804431f7e4e646fa656ff81d89175b75bf50f959
Oct 31 03:39:46.781866 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:46.781836    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"]
Oct 31 03:39:47.115843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:47.115760    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc" event={"ID":"86ae1d89-4330-42ac-ba91-977fdb19878e","Type":"ContainerStarted","Data":"506ee0ca05a26ee800752a90804431f7e4e646fa656ff81d89175b75bf50f959"}
Oct 31 03:39:49.127454 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:49.127420    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc" event={"ID":"86ae1d89-4330-42ac-ba91-977fdb19878e","Type":"ContainerStarted","Data":"05d80fa02ededda1904c7f27df88cb2f7f26367a8d56ae65f81ce2cc60419e9c"}
Oct 31 03:39:49.251730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:49.251678    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc" podStartSLOduration=1.7020290089999999 podStartE2EDuration="3.25166476s" totalImagesPullingTime="1.549635751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:39:49.250956848 +0000 UTC m=+639.084817168" watchObservedRunningTime="2026-10-31 03:39:49.25166476 +0000 UTC m=+639.085525065"
Oct 31 03:39:49.414729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:49.414645    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:50.270756 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:50.270722    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:53.393640 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:53.393609    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:56.414861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.414820    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:56.427141 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.427112    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:56.699934 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.699842    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"]
Oct 31 03:39:56.703663 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.703639    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.758067 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.758030    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 31 03:39:56.759735 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.759706    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"]
Oct 31 03:39:56.799827 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.799787    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf9ws\" (UniqueName: \"kubernetes.io/projected/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-kube-api-access-mf9ws\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.799998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.799841    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.799998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.799926    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.800102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.800014    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.800102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.800052    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.800198 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.800164    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.800647 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.800628    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:39:56.901412 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901375    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901449    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf9ws\" (UniqueName: \"kubernetes.io/projected/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-kube-api-access-mf9ws\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901489    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901520    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901564    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901588    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901919 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901896    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.901995 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901973    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.902041 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.901995    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.903965 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.903933    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.904158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.904136    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:56.975009 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:56.974919    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf9ws\" (UniqueName: \"kubernetes.io/projected/017390dc-370f-4bd0-ae0e-0340ebf8b2cb-kube-api-access-mf9ws\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-mj8ld\" (UID: \"017390dc-370f-4bd0-ae0e-0340ebf8b2cb\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:57.016688 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:57.016651    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:39:57.174102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:57.174074    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc"
Oct 31 03:39:57.403950 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:39:57.403916    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017390dc_370f_4bd0_ae0e_0340ebf8b2cb.slice/crio-3a82ee7d0963d177621a674e82fc11872ee847235e7c8f857dc8c5102cd57406 WatchSource:0}: Error finding container 3a82ee7d0963d177621a674e82fc11872ee847235e7c8f857dc8c5102cd57406: Status 404 returned error can't find the container with id 3a82ee7d0963d177621a674e82fc11872ee847235e7c8f857dc8c5102cd57406
Oct 31 03:39:57.404998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:57.404971    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"]
Oct 31 03:39:58.169713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:58.169677    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld" event={"ID":"017390dc-370f-4bd0-ae0e-0340ebf8b2cb","Type":"ContainerStarted","Data":"4b106ee614d2717be5b83ac89684dfd3d71a64c016341c72e709356ed60133fe"}
Oct 31 03:39:58.170098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:58.169736    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld" event={"ID":"017390dc-370f-4bd0-ae0e-0340ebf8b2cb","Type":"ContainerStarted","Data":"3a82ee7d0963d177621a674e82fc11872ee847235e7c8f857dc8c5102cd57406"}
Oct 31 03:39:58.295233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:58.295091    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld" podStartSLOduration=1.6839243929999999 podStartE2EDuration="2.295076731s" totalImagesPullingTime="611.152338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:39:58.293650182 +0000 UTC m=+648.127510484" watchObservedRunningTime="2026-10-31 03:39:58.295076731 +0000 UTC m=+648.128937035"
Oct 31 03:39:59.017444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:39:59.017407    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:40:00.392448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.392410    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"]
Oct 31 03:40:00.395904 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.395887    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.435296 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.435262    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f66cfd64-9d29-4813-86a2-4b0e766fae49-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.435296 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.435304    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.435546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.435383    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.435546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.435428    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.435546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.435498    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jgzp\" (UniqueName: \"kubernetes.io/projected/f66cfd64-9d29-4813-86a2-4b0e766fae49-kube-api-access-4jgzp\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.435654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.435570    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.453497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.453463    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"]
Oct 31 03:40:00.488625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.488579    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:40:00.536442 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536411    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f66cfd64-9d29-4813-86a2-4b0e766fae49-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536442 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536449    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536569    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536612    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536666    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jgzp\" (UniqueName: \"kubernetes.io/projected/f66cfd64-9d29-4813-86a2-4b0e766fae49-kube-api-access-4jgzp\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536722    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536842    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.536921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536873    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.537079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.536975    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.539270 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.539237    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f66cfd64-9d29-4813-86a2-4b0e766fae49-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.545736 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.545709    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 31 03:40:00.549874 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.549850    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f66cfd64-9d29-4813-86a2-4b0e766fae49-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.599967 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.599936    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jgzp\" (UniqueName: \"kubernetes.io/projected/f66cfd64-9d29-4813-86a2-4b0e766fae49-kube-api-access-4jgzp\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-ljbgs\" (UID: \"f66cfd64-9d29-4813-86a2-4b0e766fae49\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:00.708438 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:00.708357    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:01.085014 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:40:01.084978    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66cfd64_9d29_4813_86a2_4b0e766fae49.slice/crio-70122ccb0d26ad4200081feaf55575a59e7e232b2f96d172dd3390643d50dbaf WatchSource:0}: Error finding container 70122ccb0d26ad4200081feaf55575a59e7e232b2f96d172dd3390643d50dbaf: Status 404 returned error can't find the container with id 70122ccb0d26ad4200081feaf55575a59e7e232b2f96d172dd3390643d50dbaf
Oct 31 03:40:01.086610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:01.086586    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"]
Oct 31 03:40:01.185599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:01.185566    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs" event={"ID":"f66cfd64-9d29-4813-86a2-4b0e766fae49","Type":"ContainerStarted","Data":"70122ccb0d26ad4200081feaf55575a59e7e232b2f96d172dd3390643d50dbaf"}
Oct 31 03:40:02.192737 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:02.192635    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs" event={"ID":"f66cfd64-9d29-4813-86a2-4b0e766fae49","Type":"ContainerStarted","Data":"17a25327d75c3310eec0317e0d652eb074cefd60eb65ef0b2ef22138ae62deba"}
Oct 31 03:40:02.315574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:02.315508    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs" podStartSLOduration=1.6075502529999999 podStartE2EDuration="2.315491389s" totalImagesPullingTime="707.941136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:40:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:40:02.314309899 +0000 UTC m=+652.148170205" watchObservedRunningTime="2026-10-31 03:40:02.315491389 +0000 UTC m=+652.149351685"
Oct 31 03:40:02.708475 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:02.708431    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:06.381279 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:40:06.381242    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:40:07.017403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.017362    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:40:07.029937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.029908    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:40:07.045656 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.045619    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"]
Oct 31 03:40:07.049434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.049406    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.105885 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.105833    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"]
Oct 31 03:40:07.106168 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.106142    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.106302 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.106192    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67694bd7-6e2d-42ac-b33b-befb107864a8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.106410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.106386    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.106499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.106428    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.106499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.106449    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.106584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.106570    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrnts\" (UniqueName: \"kubernetes.io/projected/67694bd7-6e2d-42ac-b33b-befb107864a8-kube-api-access-lrnts\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.135840 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.135806    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 31 03:40:07.137780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.137750    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:40:07.207941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.207900    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.207949    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.207978    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.208020    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrnts\" (UniqueName: \"kubernetes.io/projected/67694bd7-6e2d-42ac-b33b-befb107864a8-kube-api-access-lrnts\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.208086    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.208117    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67694bd7-6e2d-42ac-b33b-befb107864a8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208431 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.208337    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208431 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.208392    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.208542 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.208482    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.210387 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.210362    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67694bd7-6e2d-42ac-b33b-befb107864a8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.210553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.210535    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67694bd7-6e2d-42ac-b33b-befb107864a8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.230090 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.230063    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-mj8ld"
Oct 31 03:40:07.271994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.271919    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrnts\" (UniqueName: \"kubernetes.io/projected/67694bd7-6e2d-42ac-b33b-befb107864a8-kube-api-access-lrnts\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c\" (UID: \"67694bd7-6e2d-42ac-b33b-befb107864a8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.362910 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.362869    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:07.735221 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:40:07.735170    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67694bd7_6e2d_42ac_b33b_befb107864a8.slice/crio-e9649e0aed897a36e1e2ff5ac4c83213cb8f56f4ab767c0a74f87548c4d99aba WatchSource:0}: Error finding container e9649e0aed897a36e1e2ff5ac4c83213cb8f56f4ab767c0a74f87548c4d99aba: Status 404 returned error can't find the container with id e9649e0aed897a36e1e2ff5ac4c83213cb8f56f4ab767c0a74f87548c4d99aba
Oct 31 03:40:07.735886 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:07.735866    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"]
Oct 31 03:40:08.224656 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:08.224612    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c" event={"ID":"67694bd7-6e2d-42ac-b33b-befb107864a8","Type":"ContainerStarted","Data":"e9649e0aed897a36e1e2ff5ac4c83213cb8f56f4ab767c0a74f87548c4d99aba"}
Oct 31 03:40:09.231753 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:09.231712    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c" event={"ID":"67694bd7-6e2d-42ac-b33b-befb107864a8","Type":"ContainerStarted","Data":"d6d3a13b98bc2e02c4241171a9c9011aef882543d3c0b660c1e824d78a69a178"}
Oct 31 03:40:09.354446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:09.354394    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c" podStartSLOduration=2.725154931 podStartE2EDuration="3.354380306s" totalImagesPullingTime="629.225375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:40:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:40:09.353905905 +0000 UTC m=+659.187766223" watchObservedRunningTime="2026-10-31 03:40:09.354380306 +0000 UTC m=+659.188240610"
Oct 31 03:40:09.363874 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:09.363837    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:10.708887 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:10.708857    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:10.721433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:10.721398    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:11.253240 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:11.253198    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-ljbgs"
Oct 31 03:40:17.240864 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.240831    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"]
Oct 31 03:40:17.248803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.248778    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.303047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.303015    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wk9x4\""
Oct 31 03:40:17.357247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.357190    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"]
Oct 31 03:40:17.363626 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.363598    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:17.377488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.377455    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:17.413999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.413960    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.413999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.413998    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414027    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414102    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414304 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414274    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414367    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcxps\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414414    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414501    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.414561 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.414529    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516058 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516017    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516064    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516148    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516172    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516201    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516264    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516315    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516420    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcxps\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516480    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516542    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516639    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.516865 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516813    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.517011 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516964    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.517125 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.516989    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.518664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.518645    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.519039 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.519022    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.577766 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.577725    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:40:17.583224 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.580314    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.583224 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.580494    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcxps\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps\") pod \"maas-default-gateway-openshift-default-7854949c9-9ndwv\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:17.862342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:17.862240    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:18.285021 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:40:18.284985    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c5b233_5328_4d32_b579_3b20a934d141.slice/crio-16f04c38e990849a069684fbc008213a49850fcd1ea2cc88831292dc64039bdb WatchSource:0}: Error finding container 16f04c38e990849a069684fbc008213a49850fcd1ea2cc88831292dc64039bdb: Status 404 returned error can't find the container with id 16f04c38e990849a069684fbc008213a49850fcd1ea2cc88831292dc64039bdb
Oct 31 03:40:18.286895 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:18.286867    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"]
Oct 31 03:40:18.287302 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:18.287282    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c"
Oct 31 03:40:19.281356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:19.281321    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" event={"ID":"99c5b233-5328-4d32-b579-3b20a934d141","Type":"ContainerStarted","Data":"16f04c38e990849a069684fbc008213a49850fcd1ea2cc88831292dc64039bdb"}
Oct 31 03:40:22.909573 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:22.909537    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:40:22.909888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:22.909607    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:40:22.909888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:22.909638    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:40:23.303196 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:23.303149    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" event={"ID":"99c5b233-5328-4d32-b579-3b20a934d141","Type":"ContainerStarted","Data":"a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f"}
Oct 31 03:40:23.481672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:23.481621    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podStartSLOduration=1.8610206599999999 podStartE2EDuration="6.481607839s" totalImagesPullingTime="4.620587179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:40:23.481018775 +0000 UTC m=+673.314879080" watchObservedRunningTime="2026-10-31 03:40:23.481607839 +0000 UTC m=+673.315468143"
Oct 31 03:40:23.866260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:23.865595    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:23.867856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:23.867823    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:23.867999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:23.867884    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:24.866232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:24.863631    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:24.866232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:24.863695    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:25.866644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:25.866603    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:25.867056 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:25.866675    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:26.866661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:26.866614    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:26.867193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:26.866677    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:27.866237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:27.865625    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:27.866237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:27.865703    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:28.864318 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:28.864265    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:28.864987 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:28.864950    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:29.863444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:29.863400    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-9ndwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 31 03:40:29.863632 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:29.863489    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 31 03:40:31.293783 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:31.293749    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:31.294290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:31.294270    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:31.294996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:31.294968    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:40:32.372706 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.372672    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"]
Oct 31 03:40:32.378381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.378360    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470469    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470680 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470514    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470680 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470552    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470756    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgjd8\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470796    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470930 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470889    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.470991 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.470942    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.471048 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.471009    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.471102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.471045    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.485707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.485671    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"]
Oct 31 03:40:32.572297 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572253    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572309    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572407    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572600    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572652    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572685    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572717    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572720    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572763    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572991 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572778    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572991 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572864    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgjd8\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.572991 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.572890    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.573129 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.573032    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.573573 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.573550    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.575515 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.575492    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.575830 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.575809    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.636518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.636439    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.636650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.636594    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgjd8\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8\") pod \"maas-default-gateway-openshift-default-7854949c9-cdnk2\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:32.691423 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:32.691385    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:33.115824 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:40:33.115783    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacb9ec4_66fa_4ed2_b074_285e281e76fb.slice/crio-8a44a682d5c8658965bc9e417730b61532d6be44b4e3c38ac3e07a97a6892442 WatchSource:0}: Error finding container 8a44a682d5c8658965bc9e417730b61532d6be44b4e3c38ac3e07a97a6892442: Status 404 returned error can't find the container with id 8a44a682d5c8658965bc9e417730b61532d6be44b4e3c38ac3e07a97a6892442
Oct 31 03:40:33.117686 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.117662    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"]
Oct 31 03:40:33.119016 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.118974    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:40:33.119097 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.119049    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:40:33.119097 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.119077    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:40:33.353272 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.353232    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" event={"ID":"dacb9ec4-66fa-4ed2-b074-285e281e76fb","Type":"ContainerStarted","Data":"3bbdf96632f0ec69161df258dd28854bc5bbfa150719be17332e2232275553cf"}
Oct 31 03:40:33.353468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.353305    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" event={"ID":"dacb9ec4-66fa-4ed2-b074-285e281e76fb","Type":"ContainerStarted","Data":"8a44a682d5c8658965bc9e417730b61532d6be44b4e3c38ac3e07a97a6892442"}
Oct 31 03:40:33.479314 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.479196    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podStartSLOduration=1.479177814 podStartE2EDuration="1.479177814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:40:33.477092951 +0000 UTC m=+683.310953251" watchObservedRunningTime="2026-10-31 03:40:33.479177814 +0000 UTC m=+683.313038113"
Oct 31 03:40:33.691843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:33.691806    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:34.693732 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:34.693693    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:34.694197 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:34.693754    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:35.692136 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:35.692098    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:35.692337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:35.692170    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:36.396026 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:40:36.395983    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:40:36.691967 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:36.691879    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:36.691967 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:36.691944    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:37.692356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:37.692324    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:37.692748 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:37.692374    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:38.692429 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:38.692396    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:38.692863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:38.692460    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:39.692331 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:39.692290    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:39.692507 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:39.692359    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:40.691777 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:40.691732    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-cdnk2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 31 03:40:40.691990 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:40.691793    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 31 03:40:41.696080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:41.696047    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:41.696507 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:41.696330    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:41.696886 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:41.696869    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 03:40:46.304012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:46.303981    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:40:46.604400 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:46.604308    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:40:52.923215 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:52.923180    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:40:53.082716 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:40:53.082684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:06.381729 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:41:06.381682    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:41:17.898638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:17.898605    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:35.202034 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:35.201998    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:35.729747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:35.729716    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:36.381786 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:41:36.381748    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:41:37.437674 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:37.437641    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:37.578127 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:37.578091    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:59.164462 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:59.164424    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:41:59.307136 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:41:59.307100    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:42:06.381336 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:42:06.381295    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:42:18.245339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:42:18.245292    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:42:36.381071 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:42:36.381032    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:43:02.549619 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:02.549586    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:02.689668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:02.689635    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:03.157007 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.156976    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"]
Oct 31 03:43:03.160799 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.160782    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.216501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.216470    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ae8e27-openshift-default-dockercfg-sgjl6\""
Oct 31 03:43:03.237002 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.236969    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:03.271621 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.271588    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"]
Oct 31 03:43:03.332398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332365    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.332652 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332622    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.332743 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332673    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.332743 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332727    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.332857 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332836    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqk5m\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.332906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332894    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.332992 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.332972    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.333056 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.333045    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.333119 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.333101    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.433718 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.433622    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.433718 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.433674    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.433963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.433719    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.433963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.433741    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.433963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.433776    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.433963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.433800    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434013    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqk5m\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434077    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434101    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434107    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434616 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434582    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434702    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434840 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434787    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.434903 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.434865    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.436179 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.436157    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.436452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.436431    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.498853 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.498818    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqk5m\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.499005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.498854    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.569655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.569616    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:03.713763 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.713674    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:03.776450 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.776413    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:03.810844 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.810812    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"]
Oct 31 03:43:03.814625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.814605    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.845340 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845042    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.845340 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845108    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.845340 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845198    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dfq8\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.846146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845738    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.846146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845780    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.846146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845841    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.846146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845865    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.846146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845898    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.846146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.845960    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.861836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.861451    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:03.880201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.875854    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-68aa3e-openshift-default-dockercfg-jktfb\""
Oct 31 03:43:03.880201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.878562    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"]
Oct 31 03:43:03.946524 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946484    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.946731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946533    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.946731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946593    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dfq8\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.946731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946634    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.946731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946657    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.946731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946692    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.946731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946709    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946760    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946816    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.946998    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.947161    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.947227    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947538 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.947518    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.947609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.947573    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.949171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.949148    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:03.949514 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:03.949496    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:04.004552 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.004517    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:04.009047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.009014    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:04.009186 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.009092    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dfq8\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:04.128858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.128823    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:04.212359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.210969    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"]
Oct 31 03:43:04.212359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.211531    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:04.212359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.211629    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:04.212359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.211665    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:04.236095 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.236067    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:04.389152 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.389120    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:04.464455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.464421    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"]
Oct 31 03:43:04.468156 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.468140    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.514007 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.513972    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"]
Oct 31 03:43:04.536284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.536174    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-e5b8f9-openshift-default-dockercfg-qt6w5\""
Oct 31 03:43:04.536534 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.536509    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:04.537668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.537630    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:04.537824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.537717    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:04.537824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.537767    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:04.537824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.537786    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"]
Oct 31 03:43:04.553672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553642    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkjwp\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.553839 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553683    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.553839 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553709    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.553839 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553813    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.554028 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553849    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.554028 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553884    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.554028 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553929    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.554028 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553979    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.554028 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.553997    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.654955 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.654920    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.654978    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkjwp\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655025    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655049    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655108    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655158    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655201    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655279    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655311    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655340    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655446    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655513    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.655998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.655713    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.656238 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.656200    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.657546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.657524    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.658263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.658241    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.678986 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.678954    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:04.718101 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.718067    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkjwp\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.720841 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.720814    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.785583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.785503    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:04.831882 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.831846    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:04.978094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:04.977996    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:05.098968 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.098869    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" event={"ID":"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2","Type":"ContainerStarted","Data":"16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"}
Oct 31 03:43:05.098968 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.098942    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" event={"ID":"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2","Type":"ContainerStarted","Data":"185c19f24a9614f08d5f71f101c2623e8fb64e665b976af51b8fe470f9ae247a"}
Oct 31 03:43:05.100481 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.100458    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" event={"ID":"82e25961-bd59-441a-9b0a-855dee1f6f47","Type":"ContainerStarted","Data":"9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"}
Oct 31 03:43:05.100597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.100502    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" event={"ID":"82e25961-bd59-441a-9b0a-855dee1f6f47","Type":"ContainerStarted","Data":"2b4cc32a69d87cf67f8fc275c4011be318e275053835e89cdb6171d5df6ef2a0"}
Oct 31 03:43:05.116655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.116612    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:05.129567 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.129535    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:05.134782 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.134755    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:05.228415 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:05.228375    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d35492_ee24_44d7_8e6b_2db34f6ac5fc.slice/crio-0d71c7150275dcc141eb885c78577140596c694739954966bc28a290ce6daaa2 WatchSource:0}: Error finding container 0d71c7150275dcc141eb885c78577140596c694739954966bc28a290ce6daaa2: Status 404 returned error can't find the container with id 0d71c7150275dcc141eb885c78577140596c694739954966bc28a290ce6daaa2
Oct 31 03:43:05.230514 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.230491    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"]
Oct 31 03:43:05.230945 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.230909    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:05.231031 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.230979    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:05.231031 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.231007    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:05.259947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.259901    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podStartSLOduration=2.259887396 podStartE2EDuration="2.259887396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:05.257735365 +0000 UTC m=+835.091595670" watchObservedRunningTime="2026-10-31 03:43:05.259887396 +0000 UTC m=+835.093747700"
Oct 31 03:43:05.385671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.385547    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podStartSLOduration=2.385530214 podStartE2EDuration="2.385530214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:05.384841086 +0000 UTC m=+835.218701391" watchObservedRunningTime="2026-10-31 03:43:05.385530214 +0000 UTC m=+835.219390518"
Oct 31 03:43:05.776637 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.776598    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:05.782500 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:05.782473    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:06.107947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.107849    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" event={"ID":"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc","Type":"ContainerStarted","Data":"69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"}
Oct 31 03:43:06.107947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.107915    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" event={"ID":"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc","Type":"ContainerStarted","Data":"0d71c7150275dcc141eb885c78577140596c694739954966bc28a290ce6daaa2"}
Oct 31 03:43:06.108193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.108079    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:06.108290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.108272    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:06.109060 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.109040    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:06.109167 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.109097    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:06.232718 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.232664    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podStartSLOduration=2.23264904 podStartE2EDuration="2.23264904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:06.231132423 +0000 UTC m=+836.064992721" watchObservedRunningTime="2026-10-31 03:43:06.23264904 +0000 UTC m=+836.066509344"
Oct 31 03:43:06.389487 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:43:06.389337    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:43:06.786316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.786282    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:06.791409 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:06.791381    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:07.113504 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:07.113414    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:07.114110 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:07.114092    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:43:07.736925 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:07.736881    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:08.286550 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:08.286481    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:08.454888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:08.454848    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:08.600480 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:08.600375    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:08.891625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:08.891520    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:09.083126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:09.083076    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:09.229397 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:09.229299    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:10.422888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:10.422860    5906 scope.go:122] "RemoveContainer" containerID="0d7014f21fac18bffe8ee6dda0f2fc18ccee35ff0be90466728fee12e715c9bc"
Oct 31 03:43:11.386764 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.386727    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:11.546968 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.546934    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:11.797599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.797570    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"]
Oct 31 03:43:11.802808 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.802786    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:11.842078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.842025    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:11.842284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.842140    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6vk2\" (UniqueName: \"kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2\") pod \"payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:11.856663 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.856631    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"]
Oct 31 03:43:11.906142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.906111    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-537e34-dockercfg-hsttt\""
Oct 31 03:43:11.907374 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.906812    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-537e34\""
Oct 31 03:43:11.943019 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.942980    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:11.943229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.943056    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6vk2\" (UniqueName: \"kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2\") pod \"payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:11.943777 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:11.943759    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:12.006413 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:12.006376    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6vk2\" (UniqueName: \"kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2\") pod \"payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:12.115801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:12.115719    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:12.489016 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:12.488976    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e944735_7b85_4160_a3ec_a4cefd28f746.slice/crio-ed7419e6427bd02f19c56893274954cae2d620d71195bf0708b373a79f60f112 WatchSource:0}: Error finding container ed7419e6427bd02f19c56893274954cae2d620d71195bf0708b373a79f60f112: Status 404 returned error can't find the container with id ed7419e6427bd02f19c56893274954cae2d620d71195bf0708b373a79f60f112
Oct 31 03:43:12.490602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:12.490578    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"]
Oct 31 03:43:12.491246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:12.491230    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:43:12.552912 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:12.552882    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:12.857444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:12.857411    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:13.049749 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:13.049711    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:13.143411 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:13.143322    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" event={"ID":"1e944735-7b85-4160-a3ec-a4cefd28f746","Type":"ContainerStarted","Data":"ed7419e6427bd02f19c56893274954cae2d620d71195bf0708b373a79f60f112"}
Oct 31 03:43:13.219629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:13.219586    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:13.392176 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:13.392138    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:13.538614 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:13.538568    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:13.686834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:13.686776    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:14.585179 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:14.585123    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:14.745097 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:14.745056    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:14.890088 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:14.890012    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:15.961811 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:15.961725    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:16.129495 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.129461    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:16.159610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.159571    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" event={"ID":"1e944735-7b85-4160-a3ec-a4cefd28f746","Type":"ContainerStarted","Data":"9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d"}
Oct 31 03:43:16.159818 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.159665    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:16.280762 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.280702    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" podStartSLOduration=2.245296841 podStartE2EDuration="5.280686763s" totalImagesPullingTime="3.035389922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:16.278553144 +0000 UTC m=+846.112413459" watchObservedRunningTime="2026-10-31 03:43:16.280686763 +0000 UTC m=+846.114547067"
Oct 31 03:43:16.556698 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.552470    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"]
Oct 31 03:43:16.560269 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.560243    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.611747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.611711    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"]
Oct 31 03:43:16.614727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.614695    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-412ba5\""
Oct 31 03:43:16.661420 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.661384    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-412ba5-dockercfg-qmjd2\""
Oct 31 03:43:16.696671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.696637    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7mx5\" (UniqueName: \"kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5\") pod \"payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.696845 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.696681    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.797763 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.797728    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7mx5\" (UniqueName: \"kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5\") pod \"payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.797763 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.797771    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.798502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.798483    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.860835 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.860755    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7mx5\" (UniqueName: \"kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5\") pod \"payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:16.873665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:16.873637    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:17.248157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:17.248125    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09711f71_2568_419f_8a52_d22a2bd745db.slice/crio-327cf1daa437df0d972b4f7e8f404647446812903608857efb75af0775906d32 WatchSource:0}: Error finding container 327cf1daa437df0d972b4f7e8f404647446812903608857efb75af0775906d32: Status 404 returned error can't find the container with id 327cf1daa437df0d972b4f7e8f404647446812903608857efb75af0775906d32
Oct 31 03:43:17.249629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:17.249604    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"]
Oct 31 03:43:18.171408 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.171373    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" event={"ID":"09711f71-2568-419f-8a52-d22a2bd745db","Type":"ContainerStarted","Data":"88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d"}
Oct 31 03:43:18.171621 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.171426    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" event={"ID":"09711f71-2568-419f-8a52-d22a2bd745db","Type":"ContainerStarted","Data":"327cf1daa437df0d972b4f7e8f404647446812903608857efb75af0775906d32"}
Oct 31 03:43:18.171621 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.171519    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:18.291948 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.291915    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:18.295980 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.295931    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" podStartSLOduration=2.295759414 podStartE2EDuration="2.295759414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:18.29430899 +0000 UTC m=+848.128169294" watchObservedRunningTime="2026-10-31 03:43:18.295759414 +0000 UTC m=+848.129619719"
Oct 31 03:43:18.697856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.697820    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"]
Oct 31 03:43:18.701618 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.701603    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:18.762508 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.762473    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"]
Oct 31 03:43:18.810455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.810426    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-68aa3e\""
Oct 31 03:43:18.810455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.810426    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-68aa3e-dockercfg-z5d66\""
Oct 31 03:43:18.821085 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.821057    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:18.821242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.821113    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc4sj\" (UniqueName: \"kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj\") pod \"payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:18.922974 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.922640    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4sj\" (UniqueName: \"kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj\") pod \"payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:18.922974 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.922812    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:18.923629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.923606    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:18.986342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:18.986271    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc4sj\" (UniqueName: \"kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj\") pod \"payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:19.014044 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:19.014011    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:19.343635 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:19.343588    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:19.415927 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:19.415886    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1acc431a_e60d_48e7_bc7e_035320647d1f.slice/crio-960df353efc0ea8e10d9014b2d910767619940a7ea36d66d1292a8ea7f006dfc WatchSource:0}: Error finding container 960df353efc0ea8e10d9014b2d910767619940a7ea36d66d1292a8ea7f006dfc: Status 404 returned error can't find the container with id 960df353efc0ea8e10d9014b2d910767619940a7ea36d66d1292a8ea7f006dfc
Oct 31 03:43:19.417159 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:19.417133    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"]
Oct 31 03:43:19.498529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:19.498502    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:19.637387 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:19.637288    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:19.858330 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:19.858289    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:20.184489 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:20.184454    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" event={"ID":"1acc431a-e60d-48e7-bc7e-035320647d1f","Type":"ContainerStarted","Data":"691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e"}
Oct 31 03:43:20.184675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:20.184503    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" event={"ID":"1acc431a-e60d-48e7-bc7e-035320647d1f","Type":"ContainerStarted","Data":"960df353efc0ea8e10d9014b2d910767619940a7ea36d66d1292a8ea7f006dfc"}
Oct 31 03:43:20.184675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:20.184555    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:20.305027 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:20.304980    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" podStartSLOduration=2.304937551 podStartE2EDuration="2.304937551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:20.303519338 +0000 UTC m=+850.137379643" watchObservedRunningTime="2026-10-31 03:43:20.304937551 +0000 UTC m=+850.138797856"
Oct 31 03:43:20.704630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:20.704599    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:20.872465 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:20.872430    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:21.119251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.119224    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"]
Oct 31 03:43:21.123011 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.122994    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.230502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.230469    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-ae8e27\""
Oct 31 03:43:21.230826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.230809    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-ae8e27-dockercfg-cfm8p\""
Oct 31 03:43:21.233730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.233387    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"]
Oct 31 03:43:21.247771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.247459    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svqbr\" (UniqueName: \"kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr\") pod \"payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.247771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.247516    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.248284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.248250    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:21.348644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.348609    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svqbr\" (UniqueName: \"kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr\") pod \"payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.348844 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.348660    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.349481 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.349453    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.412806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.412733    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svqbr\" (UniqueName: \"kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr\") pod \"payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.416370 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.416331    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:21.436413 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.436386    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:21.581139 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.581105    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:21.812494 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:21.812458    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd808397_af2b_4790_9325_b936b3a052de.slice/crio-38e5b87c48d897db4806a206293caf43a4a8961acf89c3378b9b1c78142a040b WatchSource:0}: Error finding container 38e5b87c48d897db4806a206293caf43a4a8961acf89c3378b9b1c78142a040b: Status 404 returned error can't find the container with id 38e5b87c48d897db4806a206293caf43a4a8961acf89c3378b9b1c78142a040b
Oct 31 03:43:21.814278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:21.814256    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"]
Oct 31 03:43:22.214668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:22.214578    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" event={"ID":"dd808397-af2b-4790-9325-b936b3a052de","Type":"ContainerStarted","Data":"518296e96a1ae94ff7b12587db668a8e0d86bfb93f198f8e05a6df0e37ba0b5b"}
Oct 31 03:43:22.214668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:22.214636    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" event={"ID":"dd808397-af2b-4790-9325-b936b3a052de","Type":"ContainerStarted","Data":"38e5b87c48d897db4806a206293caf43a4a8961acf89c3378b9b1c78142a040b"}
Oct 31 03:43:22.214668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:22.214666    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:22.340590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:22.340536    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" podStartSLOduration=1.340523753 podStartE2EDuration="1.340523753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:22.338711489 +0000 UTC m=+852.172571793" watchObservedRunningTime="2026-10-31 03:43:22.340523753 +0000 UTC m=+852.174384092"
Oct 31 03:43:24.580427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:24.580384    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:24.732051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:24.732018    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:24.895673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:24.895579    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:27.182569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:27.182064    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 03:43:27.183595 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:27.183563    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:27.525704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:27.525674    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:27.718872 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:27.718841    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:27.898774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:27.898704    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:28.105549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:28.105509    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:28.273877 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:28.273840    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:29.179762 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.179734    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 03:43:29.612909 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.612872    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"]
Oct 31 03:43:29.616576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.616556    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.674305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.674263    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-537e34-serving-cert\""
Oct 31 03:43:29.674643 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.674616    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gc447\""
Oct 31 03:43:29.674773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.674703    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 03:43:29.679969 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.679943    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"]
Oct 31 03:43:29.764720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.764672    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.765279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.765252    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.765515 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.765478    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.765672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.765654    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx898\" (UniqueName: \"kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.800993 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.800965    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:29.877277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.866820    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.877277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.866873    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx898\" (UniqueName: \"kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.877277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.866935    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.877277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.867098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.877611 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.877317    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.877611 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:43:29.877444    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-serving-cert: secret "maas-api-e2e-worker-w3-537e34-serving-cert" not found
Oct 31 03:43:29.877611 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:43:29.877528    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls podName:521f908c-8733-4b71-af51-4f0fcd339c09 nodeName:}" failed. No retries permitted until 2026-10-31 03:43:30.377507458 +0000 UTC m=+860.211367761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls") pod "maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" (UID: "521f908c-8733-4b71-af51-4f0fcd339c09") : secret "maas-api-e2e-worker-w3-537e34-serving-cert" not found
Oct 31 03:43:29.891012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.890979    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.939318 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.939283    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx898\" (UniqueName: \"kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:29.972229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:29.971760    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:30.159287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:30.159007    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:30.487749 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:30.483164    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:30.492994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:30.492927    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\") pod \"maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:30.529821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:30.529785    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:30.935521 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:30.935479    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod521f908c_8733_4b71_af51_4f0fcd339c09.slice/crio-619f35e361b9aa242c069d63be92fcfe8eb5915e02f4fc15d310e678232c5eb4 WatchSource:0}: Error finding container 619f35e361b9aa242c069d63be92fcfe8eb5915e02f4fc15d310e678232c5eb4: Status 404 returned error can't find the container with id 619f35e361b9aa242c069d63be92fcfe8eb5915e02f4fc15d310e678232c5eb4
Oct 31 03:43:30.938030 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:30.938005    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"]
Oct 31 03:43:31.211926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:31.211837    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 03:43:31.288381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:31.288344    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" event={"ID":"521f908c-8733-4b71-af51-4f0fcd339c09","Type":"ContainerStarted","Data":"619f35e361b9aa242c069d63be92fcfe8eb5915e02f4fc15d310e678232c5eb4"}
Oct 31 03:43:33.224184 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:33.224158    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 03:43:34.629200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:34.629163    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:34.920838 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:34.920801    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:35.074246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.074192    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:35.146784 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.146752    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"]
Oct 31 03:43:35.150519 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.150499    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.214362 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.214272    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"]
Oct 31 03:43:35.245098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.243963    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.245098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.244062    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzrxk\" (UniqueName: \"kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk\") pod \"payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.263390 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.263358    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-a64fd2\""
Oct 31 03:43:35.263556 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.263425    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-a64fd2-dockercfg-szslr\""
Oct 31 03:43:35.318448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.318417    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" event={"ID":"521f908c-8733-4b71-af51-4f0fcd339c09","Type":"ContainerStarted","Data":"ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7"}
Oct 31 03:43:35.318646 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.318610    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:35.345446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.345403    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzrxk\" (UniqueName: \"kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk\") pod \"payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.345649 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.345630    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.346402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.346381    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.408548 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.408516    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzrxk\" (UniqueName: \"kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk\") pod \"payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.441521 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.441464    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" podStartSLOduration=2.9407662869999998 podStartE2EDuration="6.441447964s" totalImagesPullingTime="3.500681677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:35.440180087 +0000 UTC m=+865.274040407" watchObservedRunningTime="2026-10-31 03:43:35.441447964 +0000 UTC m=+865.275308268"
Oct 31 03:43:35.464488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.464412    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:35.842239 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:35.841980    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b2fc3a_dc93_437c_8cfc_f7727595709c.slice/crio-b7a2c9719d2bbc5cf85dbdddcf0c07e75d00a7de0d548267262c79de3b4b7010 WatchSource:0}: Error finding container b7a2c9719d2bbc5cf85dbdddcf0c07e75d00a7de0d548267262c79de3b4b7010: Status 404 returned error can't find the container with id b7a2c9719d2bbc5cf85dbdddcf0c07e75d00a7de0d548267262c79de3b4b7010
Oct 31 03:43:35.843233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:35.843192    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"]
Oct 31 03:43:36.325082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:36.325039    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" event={"ID":"c7b2fc3a-dc93-437c-8cfc-f7727595709c","Type":"ContainerStarted","Data":"3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2"}
Oct 31 03:43:36.325259 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:36.325105    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" event={"ID":"c7b2fc3a-dc93-437c-8cfc-f7727595709c","Type":"ContainerStarted","Data":"b7a2c9719d2bbc5cf85dbdddcf0c07e75d00a7de0d548267262c79de3b4b7010"}
Oct 31 03:43:36.325339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:36.325305    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:36.381989 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:43:36.381952    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:43:36.447104 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:36.447033    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" podStartSLOduration=1.447020676 podStartE2EDuration="1.447020676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:36.445689565 +0000 UTC m=+866.279549869" watchObservedRunningTime="2026-10-31 03:43:36.447020676 +0000 UTC m=+866.280881031"
Oct 31 03:43:36.486579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:36.486535    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:59786->10.134.0.41:15021: read: connection reset by peer" start-of-body=
Oct 31 03:43:36.486711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:36.486597    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": read tcp 10.134.0.2:59786->10.134.0.41:15021: read: connection reset by peer"
Oct 31 03:43:37.133876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.133825    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:37.331953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.331920    5906 generic.go:356] "Generic (PLEG): container finished" podID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerID="9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317" exitCode=0
Oct 31 03:43:37.332140 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.331993    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" event={"ID":"82e25961-bd59-441a-9b0a-855dee1f6f47","Type":"ContainerDied","Data":"9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"}
Oct 31 03:43:37.332467 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.332449    5906 scope.go:122] "RemoveContainer" containerID="9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"
Oct 31 03:43:37.333641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.333617    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:37.333705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.333687    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:37.333741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.333719    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:37.343483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.343463    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:37.692327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.691257    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"]
Oct 31 03:43:37.695101 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.695075    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:37.752754 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.752719    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"]
Oct 31 03:43:37.753314 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.753235    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-e5b8f9-dockercfg-7zhbz\""
Oct 31 03:43:37.754337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.754312    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-e5b8f9\""
Oct 31 03:43:37.774040 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.773983    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:37.774255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.774160    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95dxg\" (UniqueName: \"kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg\") pod \"payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:37.875106 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.875061    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95dxg\" (UniqueName: \"kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg\") pod \"payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:37.875359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.875164    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:37.875910 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.875888    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:37.939930 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:37.939895    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95dxg\" (UniqueName: \"kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg\") pod \"payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:38.010955 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:38.010920    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:38.340012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:38.339925    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" event={"ID":"82e25961-bd59-441a-9b0a-855dee1f6f47","Type":"ContainerStarted","Data":"ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb"}
Oct 31 03:43:38.433683 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:38.433645    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9d19d73_37f6_4667_87c5_7aa6d8afc700.slice/crio-6020b8977b8576b97b6db99fb6c441264d011abfbc3841b73087f9bde7f2ee99 WatchSource:0}: Error finding container 6020b8977b8576b97b6db99fb6c441264d011abfbc3841b73087f9bde7f2ee99: Status 404 returned error can't find the container with id 6020b8977b8576b97b6db99fb6c441264d011abfbc3841b73087f9bde7f2ee99
Oct 31 03:43:38.435597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:38.435161    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"]
Oct 31 03:43:39.129359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.129309    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:39.129618 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.129594    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 03:43:39.129682 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.129641    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 03:43:39.372230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.367361    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" event={"ID":"b9d19d73-37f6-4667-87c5-7aa6d8afc700","Type":"ContainerStarted","Data":"4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6"}
Oct 31 03:43:39.372230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.367421    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" event={"ID":"b9d19d73-37f6-4667-87c5-7aa6d8afc700","Type":"ContainerStarted","Data":"6020b8977b8576b97b6db99fb6c441264d011abfbc3841b73087f9bde7f2ee99"}
Oct 31 03:43:39.372230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.367470    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:39.490291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:39.490009    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" podStartSLOduration=2.489986661 podStartE2EDuration="2.489986661s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:39.486724281 +0000 UTC m=+869.320584579" watchObservedRunningTime="2026-10-31 03:43:39.489986661 +0000 UTC m=+869.323846965"
Oct 31 03:43:40.130102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:40.130067    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 03:43:40.130304 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:40.130120    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 03:43:40.295423 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:40.295295    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:41.129185 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:41.129146    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 03:43:41.129632 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:41.129201    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 03:43:41.330570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:41.330539    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 03:43:42.130010 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:42.129973    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 03:43:42.130554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:42.130522    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 03:43:43.129414 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:43.129373    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 31 03:43:43.129603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:43.129439    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 31 03:43:44.133823 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:44.133789    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:44.134308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:44.134097    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:44.134831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:44.134809    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 03:43:45.404161 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:45.404125    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:45.593872 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:45.593842    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:45.743297 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:45.743248    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:46.932634 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:46.932554    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:47.081070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.081031    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:47.335254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.335191    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 03:43:47.434290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.434252    5906 generic.go:356] "Generic (PLEG): container finished" podID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerID="16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3" exitCode=0
Oct 31 03:43:47.434467 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.434317    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" event={"ID":"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2","Type":"ContainerDied","Data":"16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"}
Oct 31 03:43:47.434695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.434675    5906 scope.go:122] "RemoveContainer" containerID="16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"
Oct 31 03:43:47.435984 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.435954    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:47.436120 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.436037    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:47.436120 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:47.436084    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:43:48.445237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:48.441796    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" event={"ID":"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2","Type":"ContainerStarted","Data":"5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953"}
Oct 31 03:43:48.699248 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:48.699141    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:48.776931 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:48.776891    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:48.778137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:48.778109    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 03:43:48.778292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:48.778166    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 03:43:49.367548 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:49.367510    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:49.533411 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:49.533379    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:49.776952 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:49.776917    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 03:43:49.777149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:49.776971    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 03:43:50.775670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:50.775613    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:50.776725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:50.776696    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 03:43:50.776875 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:50.776750    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 03:43:51.109249 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:51.109131    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:51.397076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:51.397004    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 03:43:51.777759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:51.777727    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 03:43:51.778197 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:51.777783    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 03:43:52.777557 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:52.777516    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 31 03:43:52.777788 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:52.777569    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 31 03:43:52.778616 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:52.778568    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:53.781333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:53.781296    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:53.782309 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:53.782293    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 03:43:55.309244 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.309188    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"]
Oct 31 03:43:55.312072 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.312050    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.342498 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.342461    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.342677 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.342627    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.342740 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.342680    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfm9s\" (UniqueName: \"kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.342803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.342774    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.418280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.418243    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-a64fd2-serving-cert\""
Oct 31 03:43:55.420320 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.420294    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"]
Oct 31 03:43:55.444199 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.444152    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.444395 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.444267    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfm9s\" (UniqueName: \"kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.444395 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.444346    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.444493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.444405    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.444608 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:43:55.444590    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-serving-cert: secret "maas-api-e2e-worker-w2-a64fd2-serving-cert" not found
Oct 31 03:43:55.444729 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:43:55.444709    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls podName:7a41ca42-cd82-4aaa-b6f7-33c2df136fdb nodeName:}" failed. No retries permitted until 2026-10-31 03:43:55.944673941 +0000 UTC m=+885.778534237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls") pod "maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" (UID: "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb") : secret "maas-api-e2e-worker-w2-a64fd2-serving-cert" not found
Oct 31 03:43:55.445121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.445098    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.446949 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.446921    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.485040 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.484997    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:55.537368 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.537334    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfm9s\" (UniqueName: \"kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.779579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.779532    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:55.916722 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.916684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:55.949126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.949097    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:55.951464 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:55.951444    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:56.225124 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:56.225036    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:56.597653 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:43:56.597621    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a41ca42_cd82_4aaa_b6f7_33c2df136fdb.slice/crio-026a96fc31b4bcf9d693b8ac7b853c0a19beb651653a4262a2cdf2e26c0a8c64 WatchSource:0}: Error finding container 026a96fc31b4bcf9d693b8ac7b853c0a19beb651653a4262a2cdf2e26c0a8c64: Status 404 returned error can't find the container with id 026a96fc31b4bcf9d693b8ac7b853c0a19beb651653a4262a2cdf2e26c0a8c64
Oct 31 03:43:56.598645 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:56.598621    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"]
Oct 31 03:43:56.624584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:56.624540    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:43:57.520098 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:57.520060    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" event={"ID":"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb","Type":"ContainerStarted","Data":"026a96fc31b4bcf9d693b8ac7b853c0a19beb651653a4262a2cdf2e26c0a8c64"}
Oct 31 03:43:58.526951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:58.526912    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" event={"ID":"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb","Type":"ContainerStarted","Data":"1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5"}
Oct 31 03:43:58.527447 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:58.526971    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:43:58.649164 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:43:58.649099    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" podStartSLOduration=2.671378001 podStartE2EDuration="3.649075625s" totalImagesPullingTime="977.697624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:58.647970774 +0000 UTC m=+888.481831087" watchObservedRunningTime="2026-10-31 03:43:58.649075625 +0000 UTC m=+888.482935930"
Oct 31 03:44:00.208569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.208532    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"]
Oct 31 03:44:00.212419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.212398    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.300116 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.300075    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.300322 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.300247    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.300322 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.300293    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpp6f\" (UniqueName: \"kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.300469 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.300442    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.317947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.317916    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-e5b8f9-serving-cert\""
Oct 31 03:44:00.319925 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.319900    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"]
Oct 31 03:44:00.384124 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.384088    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:00.401416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.401378    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.401592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.401487    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.401658 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.401590    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.401658 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.401636    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp6f\" (UniqueName: \"kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.401761 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:44:00.401739    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-serving-cert: secret "maas-api-e2e-worker-w5-e5b8f9-serving-cert" not found
Oct 31 03:44:00.401811 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:44:00.401804    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls podName:f0f60323-8f9e-41ec-bea7-bdd84b92119c nodeName:}" failed. No retries permitted until 2026-10-31 03:44:00.901787379 +0000 UTC m=+890.735647678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls") pod "maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" (UID: "f0f60323-8f9e-41ec-bea7-bdd84b92119c") : secret "maas-api-e2e-worker-w5-e5b8f9-serving-cert" not found
Oct 31 03:44:00.402251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.402231    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.403867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.403846    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.464830 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.464754    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpp6f\" (UniqueName: \"kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.572482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.572447    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:00.713294 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.713260    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:00.907889 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.907856    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:00.910230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:00.910178    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\") pod \"maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:01.126549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:01.126518    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:01.502360 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:44:01.502305    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f60323_8f9e_41ec_bea7_bdd84b92119c.slice/crio-ba1762ac30420a4a5ea5d386a45befb711a9e5db7eb0322deaf605e79c714362 WatchSource:0}: Error finding container ba1762ac30420a4a5ea5d386a45befb711a9e5db7eb0322deaf605e79c714362: Status 404 returned error can't find the container with id ba1762ac30420a4a5ea5d386a45befb711a9e5db7eb0322deaf605e79c714362
Oct 31 03:44:01.504489 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:01.504465    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"]
Oct 31 03:44:01.545856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:01.545819    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" event={"ID":"f0f60323-8f9e-41ec-bea7-bdd84b92119c","Type":"ContainerStarted","Data":"ba1762ac30420a4a5ea5d386a45befb711a9e5db7eb0322deaf605e79c714362"}
Oct 31 03:44:02.553115 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:02.553080    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" event={"ID":"f0f60323-8f9e-41ec-bea7-bdd84b92119c","Type":"ContainerStarted","Data":"d78639952624a05ed17a4e7e2f6434b9286c235aaaba22f9d3298da31f9f1e86"}
Oct 31 03:44:02.553553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:02.553217    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:02.689260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:02.689193    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" podStartSLOduration=1.7424929489999998 podStartE2EDuration="2.689179283s" totalImagesPullingTime="946.686334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:44:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:44:02.688039471 +0000 UTC m=+892.521899777" watchObservedRunningTime="2026-10-31 03:44:02.689179283 +0000 UTC m=+892.523039587"
Oct 31 03:44:04.538899 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:04.538867    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 03:44:05.390519 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:05.390486    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:06.382169 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:44:06.382132    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:44:07.276543 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:07.276502    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:07.457812 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:07.457727    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:07.595278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:07.595164    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:08.571950 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:08.571920    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 03:44:10.040739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:10.040701    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:15.041533 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:15.041496    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:16.632549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:16.632520    5906 generic.go:356] "Generic (PLEG): container finished" podID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerID="69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca" exitCode=0
Oct 31 03:44:16.632961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:16.632574    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" event={"ID":"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc","Type":"ContainerDied","Data":"69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"}
Oct 31 03:44:16.632961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:16.632954    5906 scope.go:122] "RemoveContainer" containerID="69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"
Oct 31 03:44:16.634385 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:16.634351    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:44:16.634515 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:16.634425    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:44:16.634515 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:16.634459    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:44:17.334690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:17.334650    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:17.639443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:17.639359    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" event={"ID":"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc","Type":"ContainerStarted","Data":"8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4"}
Oct 31 03:44:17.786450 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:17.786416    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:44:17.787503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:17.787475    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 31 03:44:17.787634 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:17.787525    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 31 03:44:18.786630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:18.786548    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 31 03:44:18.786630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:18.786616    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 31 03:44:18.931735 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:18.931684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:19.364761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:19.364721    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:19.786948 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:19.786910    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 31 03:44:19.787406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:19.786969    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 31 03:44:20.020714 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:20.020677    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:20.177941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:20.177863    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:20.320577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:20.320545    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:20.785836 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:20.785799    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 31 03:44:20.786047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:20.785863    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 31 03:44:21.786368 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:21.786334    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 31 03:44:21.786883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:21.786395    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 31 03:44:22.113584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:22.113503    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:44:22.593660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:22.593624    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:22.954621 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:22.954531    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:44:22.956321 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:22.956297    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 03:44:23.032915 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:23.032881    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:23.208267 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:23.208145    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:23.354062 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:23.354024    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:24.807579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:24.807542    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:29.314940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:29.314908    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:34.036996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:34.036963    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:34.507713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:34.507658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:35.175998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:35.175966    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:35.317288 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:35.317251    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:35.464227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:35.464134    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:36.381821 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:44:36.381778    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:44:38.638263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:38.638201    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:38.798842 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:38.798809    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:41.971926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:41.971888    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:43.610457 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:43.610423    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:47.502670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:47.502627    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:48.127507 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:48.127473    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:48.265742 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:48.265707    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:48.412021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:48.411936    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:49.823889 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:49.823855    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:52.604947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:52.604912    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:52.747158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:52.747122    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:52.893558 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:52.893470    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:53.036908 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:53.036874    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:54.929352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:54.929319    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:44:57.561936 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:44:57.561900    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:00.181360 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.181318    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"]
Oct 31 03:45:00.185603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.185576    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.210627 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.210598    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"]
Oct 31 03:45:00.214541 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.214521    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.239701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.239674    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 03:45:00.261650 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.261611    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b9cv\" (UniqueName: \"kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.261831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.261736    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.261831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.261768    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.261928 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.261840    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwkc4\" (UniqueName: \"kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.276983 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.276929    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"]
Oct 31 03:45:00.323607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.323577    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"]
Oct 31 03:45:00.363638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.363608    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b9cv\" (UniqueName: \"kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.363807 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.363676    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.363807 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.363703    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.363807 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.363751    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwkc4\" (UniqueName: \"kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.364540 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.364517    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.364633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.364517    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.426627 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.426598    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwkc4\" (UniqueName: \"kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.426627 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.426615    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b9cv\" (UniqueName: \"kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.499471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.499444    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:00.527787 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.527756    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:00.876609 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:45:00.876578    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe5ab632_4fc5_48ef_a616_345ea79edfc9.slice/crio-ed9ece0cffbb7d4fc17f07158cfa1b84ca9159df3e87a16fe67673b80739911e WatchSource:0}: Error finding container ed9ece0cffbb7d4fc17f07158cfa1b84ca9159df3e87a16fe67673b80739911e: Status 404 returned error can't find the container with id ed9ece0cffbb7d4fc17f07158cfa1b84ca9159df3e87a16fe67673b80739911e
Oct 31 03:45:00.878505 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.878481    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"]
Oct 31 03:45:00.909310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.909272    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" event={"ID":"be5ab632-4fc5-48ef-a616-345ea79edfc9","Type":"ContainerStarted","Data":"ed9ece0cffbb7d4fc17f07158cfa1b84ca9159df3e87a16fe67673b80739911e"}
Oct 31 03:45:00.921461 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:45:00.921431    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c35578_ee42_4992_a608_50aae87b97c1.slice/crio-2b2e415ece69ac77a171dc43fd79aa638a6ac2c15f73efabd19318e6fbdd4091 WatchSource:0}: Error finding container 2b2e415ece69ac77a171dc43fd79aa638a6ac2c15f73efabd19318e6fbdd4091: Status 404 returned error can't find the container with id 2b2e415ece69ac77a171dc43fd79aa638a6ac2c15f73efabd19318e6fbdd4091
Oct 31 03:45:00.923837 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:00.923817    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"]
Oct 31 03:45:01.915268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:01.915231    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerStarted","Data":"2b2e415ece69ac77a171dc43fd79aa638a6ac2c15f73efabd19318e6fbdd4091"}
Oct 31 03:45:02.011691 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:02.011658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:05.939019 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:05.938986    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerStarted","Data":"a5c8eaea9a0e51cb48179fc5a9465b057666198e10fb71c86448af79df9ff880"}
Oct 31 03:45:06.059437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.059391    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" podStartSLOduration=1.8757616179999999 podStartE2EDuration="6.059376601s" totalImagesPullingTime="4.183614983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:45:06.057092388 +0000 UTC m=+955.890952694" watchObservedRunningTime="2026-10-31 03:45:06.059376601 +0000 UTC m=+955.893236905"
Oct 31 03:45:06.387874 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:06.387784    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:45:06.542600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.542547    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:06.876669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.876625    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:06.950628 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.950547    5906 generic.go:356] "Generic (PLEG): container finished" podID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerID="4c7e40897747775690e21d70d0743a157145b9a94d5ee5650abdefad732f789c" exitCode=60
Oct 31 03:45:06.951074 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.950635    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" event={"ID":"be5ab632-4fc5-48ef-a616-345ea79edfc9","Type":"ContainerDied","Data":"4c7e40897747775690e21d70d0743a157145b9a94d5ee5650abdefad732f789c"}
Oct 31 03:45:06.951074 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.950899    5906 scope.go:122] "RemoveContainer" containerID="4c7e40897747775690e21d70d0743a157145b9a94d5ee5650abdefad732f789c"
Oct 31 03:45:06.952390 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.952359    5906 generic.go:356] "Generic (PLEG): container finished" podID="55c35578-ee42-4992-a608-50aae87b97c1" containerID="a5c8eaea9a0e51cb48179fc5a9465b057666198e10fb71c86448af79df9ff880" exitCode=60
Oct 31 03:45:06.952524 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.952401    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerDied","Data":"a5c8eaea9a0e51cb48179fc5a9465b057666198e10fb71c86448af79df9ff880"}
Oct 31 03:45:06.952675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:06.952661    5906 scope.go:122] "RemoveContainer" containerID="a5c8eaea9a0e51cb48179fc5a9465b057666198e10fb71c86448af79df9ff880"
Oct 31 03:45:07.960338 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:07.960302    5906 generic.go:356] "Generic (PLEG): container finished" podID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerID="86fbf2b293d54944c438fd39f449cb808dc42ed5b40f94e2d419330ec1259ce1" exitCode=60
Oct 31 03:45:07.960767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:07.960380    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" event={"ID":"be5ab632-4fc5-48ef-a616-345ea79edfc9","Type":"ContainerDied","Data":"86fbf2b293d54944c438fd39f449cb808dc42ed5b40f94e2d419330ec1259ce1"}
Oct 31 03:45:07.960767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:07.960450    5906 scope.go:122] "RemoveContainer" containerID="4c7e40897747775690e21d70d0743a157145b9a94d5ee5650abdefad732f789c"
Oct 31 03:45:07.960767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:07.960708    5906 scope.go:122] "RemoveContainer" containerID="86fbf2b293d54944c438fd39f449cb808dc42ed5b40f94e2d419330ec1259ce1"
Oct 31 03:45:07.961029 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:07.961005    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv_odh-ai-gateway-infra(be5ab632-4fc5-48ef-a616-345ea79edfc9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9"
Oct 31 03:45:07.962646 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:07.962621    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerStarted","Data":"640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"}
Oct 31 03:45:07.962920 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:07.962908    5906 scope.go:122] "RemoveContainer" containerID="640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"
Oct 31 03:45:07.963178 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:07.963161    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz_odh-ai-gateway-infra(55c35578-ee42-4992-a608-50aae87b97c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" podUID="55c35578-ee42-4992-a608-50aae87b97c1"
Oct 31 03:45:08.969350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:08.969316    5906 scope.go:122] "RemoveContainer" containerID="86fbf2b293d54944c438fd39f449cb808dc42ed5b40f94e2d419330ec1259ce1"
Oct 31 03:45:08.969830 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:08.969610    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv_odh-ai-gateway-infra(be5ab632-4fc5-48ef-a616-345ea79edfc9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9"
Oct 31 03:45:08.970875 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:08.970850    5906 generic.go:356] "Generic (PLEG): container finished" podID="55c35578-ee42-4992-a608-50aae87b97c1" containerID="640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962" exitCode=60
Oct 31 03:45:08.970983 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:08.970896    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerDied","Data":"640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"}
Oct 31 03:45:08.970983 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:08.970941    5906 scope.go:122] "RemoveContainer" containerID="a5c8eaea9a0e51cb48179fc5a9465b057666198e10fb71c86448af79df9ff880"
Oct 31 03:45:08.971163 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:08.971145    5906 scope.go:122] "RemoveContainer" containerID="640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"
Oct 31 03:45:08.971429 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:08.971412    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz_odh-ai-gateway-infra(55c35578-ee42-4992-a608-50aae87b97c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" podUID="55c35578-ee42-4992-a608-50aae87b97c1"
Oct 31 03:45:09.979705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:09.979680    5906 scope.go:122] "RemoveContainer" containerID="640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"
Oct 31 03:45:09.980100 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:09.979944    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz_odh-ai-gateway-infra(55c35578-ee42-4992-a608-50aae87b97c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" podUID="55c35578-ee42-4992-a608-50aae87b97c1"
Oct 31 03:45:11.037491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:11.037457    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:15.589550 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:15.589500    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:15.925575 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:15.925491    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:18.345220 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:18.345170    5906 scope.go:122] "RemoveContainer" containerID="640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"
Oct 31 03:45:18.954311 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:18.954235    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:19.031744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:19.031649    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerStarted","Data":"bbafa470f98647468b60aa48cdac3eab8c2ce7e3ad43954c0cfedd9e4b5e2dc9"}
Oct 31 03:45:19.267577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:19.267542    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:19.344619 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:19.344593    5906 scope.go:122] "RemoveContainer" containerID="86fbf2b293d54944c438fd39f449cb808dc42ed5b40f94e2d419330ec1259ce1"
Oct 31 03:45:19.362749 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:19.362720    5906 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55c35578_ee42_4992_a608_50aae87b97c1.slice/crio-bbafa470f98647468b60aa48cdac3eab8c2ce7e3ad43954c0cfedd9e4b5e2dc9.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 03:45:19.791035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:19.790994    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:20.004323 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.004269    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:20.038872 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.038836    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" event={"ID":"be5ab632-4fc5-48ef-a616-345ea79edfc9","Type":"ContainerStarted","Data":"0c2501628d71bc1d239f032053d9eab0033c4c3f728ddc8ece718823815361c9"}
Oct 31 03:45:20.046582 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.046543    5906 generic.go:356] "Generic (PLEG): container finished" podID="55c35578-ee42-4992-a608-50aae87b97c1" containerID="bbafa470f98647468b60aa48cdac3eab8c2ce7e3ad43954c0cfedd9e4b5e2dc9" exitCode=60
Oct 31 03:45:20.047000 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.046890    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerDied","Data":"bbafa470f98647468b60aa48cdac3eab8c2ce7e3ad43954c0cfedd9e4b5e2dc9"}
Oct 31 03:45:20.047251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.047226    5906 scope.go:122] "RemoveContainer" containerID="640346cd77a9bdd8a9afaf9c97ff5dcf36c7136616c4825cd5acb5f79f8bc962"
Oct 31 03:45:20.047973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.047952    5906 scope.go:122] "RemoveContainer" containerID="bbafa470f98647468b60aa48cdac3eab8c2ce7e3ad43954c0cfedd9e4b5e2dc9"
Oct 31 03:45:20.048488 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:20.048458    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz_odh-ai-gateway-infra(55c35578-ee42-4992-a608-50aae87b97c1)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" podUID="55c35578-ee42-4992-a608-50aae87b97c1"
Oct 31 03:45:20.160243 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.160197    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"]
Oct 31 03:45:20.161638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.161598    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" podStartSLOduration=15.269727763 podStartE2EDuration="20.161584474s" totalImagesPullingTime="4.891856711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:45:20.161329247 +0000 UTC m=+969.995189552" watchObservedRunningTime="2026-10-31 03:45:20.161584474 +0000 UTC m=+969.995444756"
Oct 31 03:45:20.302008 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.301928    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"]
Oct 31 03:45:20.305944 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.305921    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.362805 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.362758    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-412ba5-openshift-default-dockercfg-dglp9\""
Oct 31 03:45:20.365929 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.365893    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"]
Oct 31 03:45:20.407507 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.407469    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.407896 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.407856    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.407922    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.408045    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.408139    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.408196    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.408476    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8rt4\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408742 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.408719    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.408784 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.408761    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.509876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.509837    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8rt4\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.509914    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.509932    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.509962    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510019    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510057    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510084    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510278    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510320    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510488    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510647 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510542    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510647 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510579    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510735 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510709    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.510779 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.510765    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.512640 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.512617    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.512855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.512836    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.573012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.572929    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.573165 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.573113    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8rt4\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:20.619863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:20.619824    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:21.042254 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:45:21.042195    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4178a3ca_62e0_4436_889d_caa4c2153905.slice/crio-9921c1765d3396e6012896ea3206f4375df603927a997057f3f99c60722e3f84 WatchSource:0}: Error finding container 9921c1765d3396e6012896ea3206f4375df603927a997057f3f99c60722e3f84: Status 404 returned error can't find the container with id 9921c1765d3396e6012896ea3206f4375df603927a997057f3f99c60722e3f84
Oct 31 03:45:21.044154 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.044118    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"]
Oct 31 03:45:21.045108 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.045080    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:45:21.045176 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.045141    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:45:21.045176 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.045169    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:45:21.054156 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.054131    5906 generic.go:356] "Generic (PLEG): container finished" podID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerID="0c2501628d71bc1d239f032053d9eab0033c4c3f728ddc8ece718823815361c9" exitCode=60
Oct 31 03:45:21.054280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.054214    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" event={"ID":"be5ab632-4fc5-48ef-a616-345ea79edfc9","Type":"ContainerDied","Data":"0c2501628d71bc1d239f032053d9eab0033c4c3f728ddc8ece718823815361c9"}
Oct 31 03:45:21.054332 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.054286    5906 scope.go:122] "RemoveContainer" containerID="86fbf2b293d54944c438fd39f449cb808dc42ed5b40f94e2d419330ec1259ce1"
Oct 31 03:45:21.056260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.054685    5906 scope.go:122] "RemoveContainer" containerID="0c2501628d71bc1d239f032053d9eab0033c4c3f728ddc8ece718823815361c9"
Oct 31 03:45:21.056260 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:21.054977    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv_odh-ai-gateway-infra(be5ab632-4fc5-48ef-a616-345ea79edfc9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9"
Oct 31 03:45:21.061729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.061706    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" event={"ID":"4178a3ca-62e0-4436-889d-caa4c2153905","Type":"ContainerStarted","Data":"9921c1765d3396e6012896ea3206f4375df603927a997057f3f99c60722e3f84"}
Oct 31 03:45:21.173555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.173522    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"]
Oct 31 03:45:21.198463 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.198436    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:21.317147 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.317066    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4\" (UniqueName: \"kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4\") pod \"55c35578-ee42-4992-a608-50aae87b97c1\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") "
Oct 31 03:45:21.317329 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.317143    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca\") pod \"55c35578-ee42-4992-a608-50aae87b97c1\" (UID: \"55c35578-ee42-4992-a608-50aae87b97c1\") "
Oct 31 03:45:21.317545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.317524    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca" pod "55c35578-ee42-4992-a608-50aae87b97c1" (UID: "55c35578-ee42-4992-a608-50aae87b97c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:45:21.319410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.319383    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4" pod "55c35578-ee42-4992-a608-50aae87b97c1" (UID: "55c35578-ee42-4992-a608-50aae87b97c1"). InnerVolumeSpecName "kube-api-access-jwkc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:45:21.418379 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.418327    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwkc4\" (UniqueName: \"kubernetes.io/projected/55c35578-ee42-4992-a608-50aae87b97c1-kube-api-access-jwkc4\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:45:21.418379 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.418360    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55c35578-ee42-4992-a608-50aae87b97c1-maas-api-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:45:21.807024 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:21.806990    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:22.073116 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.073002    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" event={"ID":"4178a3ca-62e0-4436-889d-caa4c2153905","Type":"ContainerStarted","Data":"f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d"}
Oct 31 03:45:22.076099 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.076076    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"
Oct 31 03:45:22.076188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.076113    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz" event={"ID":"55c35578-ee42-4992-a608-50aae87b97c1","Type":"ContainerDied","Data":"2b2e415ece69ac77a171dc43fd79aa638a6ac2c15f73efabd19318e6fbdd4091"}
Oct 31 03:45:22.076188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.076170    5906 scope.go:122] "RemoveContainer" containerID="bbafa470f98647468b60aa48cdac3eab8c2ce7e3ad43954c0cfedd9e4b5e2dc9"
Oct 31 03:45:22.198093 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.198034    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podStartSLOduration=2.198019266 podStartE2EDuration="2.198019266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:45:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:45:22.196390124 +0000 UTC m=+972.030250453" watchObservedRunningTime="2026-10-31 03:45:22.198019266 +0000 UTC m=+972.031879623"
Oct 31 03:45:22.238240 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.238196    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:22.329863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.329772    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca\") pod \"be5ab632-4fc5-48ef-a616-345ea79edfc9\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") "
Oct 31 03:45:22.330053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.329914    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv\" (UniqueName: \"kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv\") pod \"be5ab632-4fc5-48ef-a616-345ea79edfc9\" (UID: \"be5ab632-4fc5-48ef-a616-345ea79edfc9\") "
Oct 31 03:45:22.330193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.330165    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca" pod "be5ab632-4fc5-48ef-a616-345ea79edfc9" (UID: "be5ab632-4fc5-48ef-a616-345ea79edfc9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:45:22.330419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.330395    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be5ab632-4fc5-48ef-a616-345ea79edfc9-maas-api-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:45:22.332303 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.332269    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv" pod "be5ab632-4fc5-48ef-a616-345ea79edfc9" (UID: "be5ab632-4fc5-48ef-a616-345ea79edfc9"). InnerVolumeSpecName "kube-api-access-8b9cv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:45:22.442544 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.431684    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b9cv\" (UniqueName: \"kubernetes.io/projected/be5ab632-4fc5-48ef-a616-345ea79edfc9-kube-api-access-8b9cv\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:45:22.460660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.460616    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"]
Oct 31 03:45:22.460969 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.460943    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890305-bdbrz"]
Oct 31 03:45:22.620746 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.620639    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:22.621932 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.621903    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 03:45:22.622070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:22.621955    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 03:45:23.086041 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.086013    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv" event={"ID":"be5ab632-4fc5-48ef-a616-345ea79edfc9","Type":"ContainerDied","Data":"ed9ece0cffbb7d4fc17f07158cfa1b84ca9159df3e87a16fe67673b80739911e"}
Oct 31 03:45:23.086367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.086344    5906 scope.go:122] "RemoveContainer" containerID="0c2501628d71bc1d239f032053d9eab0033c4c3f728ddc8ece718823815361c9"
Oct 31 03:45:23.086493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.086062    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"
Oct 31 03:45:23.270309 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.270267    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"]
Oct 31 03:45:23.270576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.270548    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890305-fwmxv"]
Oct 31 03:45:23.620568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.620532    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 03:45:23.620956 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:23.620602    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 03:45:24.165232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:24.161310    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:24.372007 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:24.371962    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55c35578-ee42-4992-a608-50aae87b97c1" path="/var/lib/kubelet/pods/55c35578-ee42-4992-a608-50aae87b97c1/volumes"
Oct 31 03:45:24.372545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:24.372521    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" path="/var/lib/kubelet/pods/be5ab632-4fc5-48ef-a616-345ea79edfc9/volumes"
Oct 31 03:45:24.475953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:24.475853    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:24.621228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:24.621169    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 03:45:24.621622 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:24.621271    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 03:45:25.621088 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:25.621048    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 03:45:25.621301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:25.621112    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 03:45:26.621219 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:26.621174    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 03:45:26.621382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:26.621245    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 03:45:27.620985 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:27.620939    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 31 03:45:27.621247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:27.621005    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 31 03:45:28.625255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:28.625198    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:28.625622 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:28.625519    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:28.626443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:28.626420    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 03:45:28.989421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:28.989331    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:33.526054 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:33.526021    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:33.666404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:33.666374    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:36.382266 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:45:36.382230    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:45:38.016497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:38.016460    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:42.491313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:42.491275    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:43.167458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:43.167423    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:43.307403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:43.307365    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:47.120071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:47.120039    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:50.866567 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:50.866483    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:51.616867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:51.616828    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:45:56.093021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:45:56.092987    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:00.555669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:00.555633    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:05.087489 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:05.087452    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:06.381881 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:46:06.381839    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:46:08.147553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:08.147521    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:09.707109 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:09.707075    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:14.210012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:14.209973    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:18.783575 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:18.783540    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:19.606318 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:19.606283    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:20.025068 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.025034    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"]
Oct 31 03:46:20.025509 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.025290    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy" containerID="cri-o://a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f" gracePeriod=30
Oct 31 03:46:20.181607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.181580    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:20.205981 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.205948    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"]
Oct 31 03:46:20.206541 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206524    5906 cpu_manager.go:414] "containerMap: removing container" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:46:20.206600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206543    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:46:20.206600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206560    5906 cpu_manager.go:414] "containerMap: removing container" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:46:20.206600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206566    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:46:20.206600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206592    5906 cpu_manager.go:414] "containerMap: removing container" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:46:20.206600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206598    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:46:20.206747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206621    5906 cpu_manager.go:414] "containerMap: removing container" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:46:20.206747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206626    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:46:20.206747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206710    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:46:20.206747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206721    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:46:20.206747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206731    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:46:20.206747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.206743    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:46:20.211767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.211743    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230493    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230692 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230545    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230692 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230633    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230692 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230678    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr6vn\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230816 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230696    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230816 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230731    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230816 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230792    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230818    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.230907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.230834    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.266361 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.266310    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-537e34-openshift-default-dockercfg-5x4wl\""
Oct 31 03:46:20.271280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.271241    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"]
Oct 31 03:46:20.331880 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.331796    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr6vn\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.331880 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.331839    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.331957    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332018    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332059    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332087    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332146    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332429 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332250    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332429 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332348    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332537 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332481    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332537 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332505    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332661    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332695    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.332827 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.332806    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.334481 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.334458    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.334708 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.334694    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.396424 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.396387    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.396620 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.396531    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr6vn\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.525768 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.525732    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:20.950727 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:46:20.950696    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod234cd54a_a721_4edb_834a_01d4a9c72042.slice/crio-46ebdb2439665ff29bd488a700bd79a4927c3e62539ececf31462c718567cfa4 WatchSource:0}: Error finding container 46ebdb2439665ff29bd488a700bd79a4927c3e62539ececf31462c718567cfa4: Status 404 returned error can't find the container with id 46ebdb2439665ff29bd488a700bd79a4927c3e62539ececf31462c718567cfa4
Oct 31 03:46:20.953082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.953057    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"]
Oct 31 03:46:20.953284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.953253    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:46:20.953339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.953325    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:46:20.953384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:20.953359    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:46:21.434230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:21.429509    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" event={"ID":"234cd54a-a721-4edb-834a-01d4a9c72042","Type":"ContainerStarted","Data":"7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654"}
Oct 31 03:46:21.434230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:21.429578    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" event={"ID":"234cd54a-a721-4edb-834a-01d4a9c72042","Type":"ContainerStarted","Data":"46ebdb2439665ff29bd488a700bd79a4927c3e62539ececf31462c718567cfa4"}
Oct 31 03:46:21.525856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:21.525819    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:22.527621 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:22.527585    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 31 03:46:22.528083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:22.527641    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 31 03:46:23.291701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:23.291664    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:23.526728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:23.526691    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 31 03:46:23.526902 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:23.526746    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 31 03:46:24.526546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:24.526505    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 31 03:46:24.526970 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:24.526566    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 31 03:46:25.334804 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.334776    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:46:25.407291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407252    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407317    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407364    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407406    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407441    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407478    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407511    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407558    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.407770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407562    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.407770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.407631    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs\") pod \"99c5b233-5328-4d32-b579-3b20a934d141\" (UID: \"99c5b233-5328-4d32-b579-3b20a934d141\") "
Oct 31 03:46:25.408594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.408099    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.408594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.408131    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.408594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.408181    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.408594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.408342    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.408594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.408383    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:46:25.410257 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.410189    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:46:25.410367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.410282    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.410668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.410643    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "kube-api-access-tcxps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:46:25.410780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.410753    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token" pod "99c5b233-5328-4d32-b579-3b20a934d141" (UID: "99c5b233-5328-4d32-b579-3b20a934d141"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:46:25.476727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.476666    5906 generic.go:356] "Generic (PLEG): container finished" podID="99c5b233-5328-4d32-b579-3b20a934d141" containerID="a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f" exitCode=0
Oct 31 03:46:25.476727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.476717    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"
Oct 31 03:46:25.476727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.476728    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" event={"ID":"99c5b233-5328-4d32-b579-3b20a934d141","Type":"ContainerDied","Data":"a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f"}
Oct 31 03:46:25.476971 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.476781    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv" event={"ID":"99c5b233-5328-4d32-b579-3b20a934d141","Type":"ContainerDied","Data":"16f04c38e990849a069684fbc008213a49850fcd1ea2cc88831292dc64039bdb"}
Oct 31 03:46:25.476971 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.476812    5906 scope.go:122] "RemoveContainer" containerID="a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f"
Oct 31 03:46:25.486997 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.486974    5906 scope.go:122] "RemoveContainer" containerID="a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f"
Oct 31 03:46:25.487309 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:46:25.487286    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f\": container with ID starting with a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f not found: ID does not exist" containerID="a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f"
Oct 31 03:46:25.487368 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.487319    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f"} err="failed to get container status \"a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f\": rpc error: code = NotFound desc = could not find container \"a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f\": container with ID starting with a1ea6e33eea7cdfda8cdcec2a4f715aa1ff50c5c38140eb78e9d12ac1528956f not found: ID does not exist"
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510769    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510802    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99c5b233-5328-4d32-b579-3b20a934d141-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510819    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510832    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510845    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99c5b233-5328-4d32-b579-3b20a934d141-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510859    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcxps\" (UniqueName: \"kubernetes.io/projected/99c5b233-5328-4d32-b579-3b20a934d141-kube-api-access-tcxps\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510872    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.510921 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.510885    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99c5b233-5328-4d32-b579-3b20a934d141-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.513183 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.513133    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podStartSLOduration=5.513115496 podStartE2EDuration="5.513115496s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:46:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:46:21.565969566 +0000 UTC m=+1031.399829872" watchObservedRunningTime="2026-10-31 03:46:25.513115496 +0000 UTC m=+1035.346975799"
Oct 31 03:46:25.526427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.526391    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 31 03:46:25.526584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.526455    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 31 03:46:25.628995 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.628915    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"]
Oct 31 03:46:25.633491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:25.633465    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-9ndwv"]
Oct 31 03:46:26.372079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:26.372022    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99c5b233-5328-4d32-b579-3b20a934d141" path="/var/lib/kubelet/pods/99c5b233-5328-4d32-b579-3b20a934d141/volumes"
Oct 31 03:46:26.527117 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:26.527082    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Oct 31 03:46:26.527530 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:26.527147    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Oct 31 03:46:27.679389 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:27.678484    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:27.679389 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:27.678873    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:27.680686 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:27.680661    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 03:46:27.833523 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:27.833486    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:32.354676 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:32.354641    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:33.782580 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:33.782534    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:36.382221 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:46:36.382165    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:46:36.915018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:36.914978    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:41.587192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:41.587153    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:43.135874 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:43.135845    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:46.275645 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:46.275601    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:50.904812 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:50.904774    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:46:55.490177 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:46:55.490142    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:00.007552 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:00.007507    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:00.878393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:00.878360    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:04.604658 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:04.604626    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:06.382759 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:47:06.382717    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:47:09.314040 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:09.314004    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:14.027685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:14.027651    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:18.510021 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:18.509939    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:19.947182 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:19.947143    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:21.798945 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.798905    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"]
Oct 31 03:47:21.799440 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799425    5906 cpu_manager.go:414] "containerMap: removing container" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:47:21.799482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799440    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:47:21.799482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799462    5906 cpu_manager.go:414] "containerMap: removing container" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:47:21.799482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799468    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:47:21.799577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799494    5906 cpu_manager.go:414] "containerMap: removing container" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy"
Oct 31 03:47:21.799577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799499    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy"
Oct 31 03:47:21.799577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799572    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="99c5b233-5328-4d32-b579-3b20a934d141" containerName="istio-proxy"
Oct 31 03:47:21.799665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799581    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="55c35578-ee42-4992-a608-50aae87b97c1" containerName="cleanup"
Oct 31 03:47:21.799665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.799588    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="be5ab632-4fc5-48ef-a616-345ea79edfc9" containerName="cleanup"
Oct 31 03:47:21.804281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.804260    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.853704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.853664    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.853916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.853741    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g55lg\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.853916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.853850    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.853916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.853910    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.854102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.854014    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.854102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.854092    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.854220 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.854190    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.854308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.854292    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.854374 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.854330    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.911776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.911739    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-a64fd2-openshift-default-dockercfg-2llsl\""
Oct 31 03:47:21.914232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.914190    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"]
Oct 31 03:47:21.955226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955160    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955532 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955509    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955554    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955645    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955708    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g55lg\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955749    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955799    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955833    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.955969 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.955877    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.956152 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.956120    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.956448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.956419    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.956805 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.956611    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.956805 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.956671    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.956805 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.956729    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.958775 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.958755    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:21.959109 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:21.959090    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:22.018989 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.018957    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:22.019191 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.019167    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g55lg\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:22.120231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.120110    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:22.565083 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:47:22.565046    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d0f5af_9f20_4742_bdc2_cc71145288ae.slice/crio-f6737e5dbb20a8d0f85422f6a17d9e1512c63f27435f7370e6d184ddc2913487 WatchSource:0}: Error finding container f6737e5dbb20a8d0f85422f6a17d9e1512c63f27435f7370e6d184ddc2913487: Status 404 returned error can't find the container with id f6737e5dbb20a8d0f85422f6a17d9e1512c63f27435f7370e6d184ddc2913487
Oct 31 03:47:22.566882 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.566854    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"]
Oct 31 03:47:22.567428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.567397    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:47:22.567490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.567461    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:47:22.567524 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.567490    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:47:22.825242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.825120    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" event={"ID":"b6d0f5af-9f20-4742-bdc2-cc71145288ae","Type":"ContainerStarted","Data":"4e66fa13830f352bf5fc52840a342d7c6ea4eaf706bc907cac90454bf5fb3428"}
Oct 31 03:47:22.825242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.825188    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" event={"ID":"b6d0f5af-9f20-4742-bdc2-cc71145288ae","Type":"ContainerStarted","Data":"f6737e5dbb20a8d0f85422f6a17d9e1512c63f27435f7370e6d184ddc2913487"}
Oct 31 03:47:22.948982 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:22.948921    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podStartSLOduration=1.948904653 podStartE2EDuration="1.948904653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:47:22.948170685 +0000 UTC m=+1092.782030989" watchObservedRunningTime="2026-10-31 03:47:22.948904653 +0000 UTC m=+1092.782764958"
Oct 31 03:47:23.121158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:23.121077    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:23.122962 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:23.122934    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:47:23.123117 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:23.122988    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 31 03:47:23.219817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:23.219784    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:24.121027 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:24.120993    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:47:24.121466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:24.121046    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 31 03:47:25.121244 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:25.121189    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:47:25.121759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:25.121283    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 31 03:47:26.121023 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:26.120986    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:47:26.121228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:26.121053    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 31 03:47:27.121044 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:27.121007    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:47:27.121576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:27.121061    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 31 03:47:27.813586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:27.813544    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:28.121042 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:28.120959    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:47:28.121042 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:28.121017    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 31 03:47:29.306646 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:29.306609    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:29.308045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:29.308021    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:29.309268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:29.309242    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 03:47:32.309696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:32.309649    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:36.383338 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:47:36.383298    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:47:36.885538 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:36.885500    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:41.375349 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:41.375313    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:45.920799 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:45.920762    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:46.274233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:46.274177    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:50.408010 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:50.407978    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:54.900999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:54.900958    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:47:59.598273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:47:59.598226    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:04.108776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:04.108739    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:06.383181 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:48:06.383144    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:48:08.736479 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:08.736444    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:09.680543 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:09.680488    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:13.486542 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.486506    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:13.753867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.753833    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"]
Oct 31 03:48:13.758113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.758093    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:13.812920 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.812882    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"]
Oct 31 03:48:13.861715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.861681    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 03:48:13.861879 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.861740    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 03:48:13.905615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.905577    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkf4b\" (UniqueName: \"kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:13.905810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.905754    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:13.905895 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:13.905871    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.006855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.006761    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.006855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.006851    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.007089 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.006891    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkf4b\" (UniqueName: \"kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.007464 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.007441    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.009365 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.009346    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.070346 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.070318    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkf4b\" (UniqueName: \"kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b\") pod \"maas-api-revoke-sub-4885f733-k4stl\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.070937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.070910    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:14.443407 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:48:14.443371    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66e2fa9_8f1a_45d8_ace5_0b54b42ad514.slice/crio-92844eae9aeeba9f0686387f99acdb404d0adf2b56954eea110e7101da6e2aa7 WatchSource:0}: Error finding container 92844eae9aeeba9f0686387f99acdb404d0adf2b56954eea110e7101da6e2aa7: Status 404 returned error can't find the container with id 92844eae9aeeba9f0686387f99acdb404d0adf2b56954eea110e7101da6e2aa7
Oct 31 03:48:14.445029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:14.444998    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"]
Oct 31 03:48:15.205242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:15.205129    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl" event={"ID":"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514","Type":"ContainerStarted","Data":"5178a1312375f27e6ebefa26312bbe48aff05adc41b91663ba86baf708b5b0a7"}
Oct 31 03:48:15.205242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:15.205188    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl" event={"ID":"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514","Type":"ContainerStarted","Data":"92844eae9aeeba9f0686387f99acdb404d0adf2b56954eea110e7101da6e2aa7"}
Oct 31 03:48:15.325600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:15.325544    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl" podStartSLOduration=2.325527556 podStartE2EDuration="2.325527556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:48:15.323856444 +0000 UTC m=+1145.157716750" watchObservedRunningTime="2026-10-31 03:48:15.325527556 +0000 UTC m=+1145.159387861"
Oct 31 03:48:16.212830 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:16.212793    5906 generic.go:356] "Generic (PLEG): container finished" podID="b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" containerID="5178a1312375f27e6ebefa26312bbe48aff05adc41b91663ba86baf708b5b0a7" exitCode=0
Oct 31 03:48:16.213299 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:16.212857    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl" event={"ID":"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514","Type":"ContainerDied","Data":"5178a1312375f27e6ebefa26312bbe48aff05adc41b91663ba86baf708b5b0a7"}
Oct 31 03:48:17.367381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.367346    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:17.442964 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.442920    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca\") pod \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") "
Oct 31 03:48:17.443279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.442995    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca\") pod \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") "
Oct 31 03:48:17.443279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.443027    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b\" (UniqueName: \"kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b\") pod \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\" (UID: \"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514\") "
Oct 31 03:48:17.443475 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.443435    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca" pod "b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" (UID: "b66e2fa9-8f1a-45d8-ace5-0b54b42ad514"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:48:17.445326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.445303    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b" pod "b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" (UID: "b66e2fa9-8f1a-45d8-ace5-0b54b42ad514"). InnerVolumeSpecName "kube-api-access-pkf4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:48:17.445685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.445661    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca" pod "b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" (UID: "b66e2fa9-8f1a-45d8-ace5-0b54b42ad514"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:48:17.545131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.545087    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:48:17.545131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.545119    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:48:17.545131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.545130    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkf4b\" (UniqueName: \"kubernetes.io/projected/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514-kube-api-access-pkf4b\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:48:17.990615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:17.990536    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:18.224797 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:18.224768    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"
Oct 31 03:48:18.224994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:18.224801    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl" event={"ID":"b66e2fa9-8f1a-45d8-ace5-0b54b42ad514","Type":"ContainerDied","Data":"92844eae9aeeba9f0686387f99acdb404d0adf2b56954eea110e7101da6e2aa7"}
Oct 31 03:48:18.224994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:18.224859    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92844eae9aeeba9f0686387f99acdb404d0adf2b56954eea110e7101da6e2aa7"
Oct 31 03:48:20.274656 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:20.274619    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:21.009390 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.009357    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"]
Oct 31 03:48:21.009868 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.009856    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" containerName="revoke-keys"
Oct 31 03:48:21.009929 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.009870    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" containerName="revoke-keys"
Oct 31 03:48:21.009986 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.009976    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" containerName="revoke-keys"
Oct 31 03:48:21.013398 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.013377    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.122699 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.122661    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"]
Oct 31 03:48:21.186279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186239    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186381    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186422    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186479    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186504    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186617 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186577    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186627    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186688 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186666    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2jxq\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.186733 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.186692    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.287501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287423    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.287501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287479    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287541    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287582    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287632    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287663    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287699    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287740    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.287788    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2jxq\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.288042    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.288084    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.288124    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.288310    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.288612 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.288494    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.289835 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.289815    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.290262 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.290247    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.350167 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.350124    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.350441 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.350421    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2jxq\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:21.626798 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:21.626701    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:22.055276 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:48:22.055242    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fb16942_2466_4501_8b09_eb90508d79c2.slice/crio-714e0e80c64e6a979f13643f7843b9204b576293fe98b3a3d5af589e759f85dc WatchSource:0}: Error finding container 714e0e80c64e6a979f13643f7843b9204b576293fe98b3a3d5af589e759f85dc: Status 404 returned error can't find the container with id 714e0e80c64e6a979f13643f7843b9204b576293fe98b3a3d5af589e759f85dc
Oct 31 03:48:22.057044 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.056970    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"]
Oct 31 03:48:22.057883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.057853    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:48:22.057947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.057916    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:48:22.057982 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.057946    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:48:22.256834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.256794    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" event={"ID":"3fb16942-2466-4501-8b09-eb90508d79c2","Type":"ContainerStarted","Data":"ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d"}
Oct 31 03:48:22.257033 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.256894    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" event={"ID":"3fb16942-2466-4501-8b09-eb90508d79c2","Type":"ContainerStarted","Data":"714e0e80c64e6a979f13643f7843b9204b576293fe98b3a3d5af589e759f85dc"}
Oct 31 03:48:22.383502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.383380    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podStartSLOduration=2.383358709 podStartE2EDuration="2.383358709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:48:22.37910637 +0000 UTC m=+1152.212966657" watchObservedRunningTime="2026-10-31 03:48:22.383358709 +0000 UTC m=+1152.217219015"
Oct 31 03:48:22.478123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.478088    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:22.634391 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:22.634302    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:23.628602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:23.628571    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 03:48:23.629035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:23.628624    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 03:48:24.627301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:24.627263    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 03:48:24.627478 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:24.627320    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 03:48:25.627317 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:25.627282    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 03:48:25.627727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:25.627355    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 03:48:26.627267 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:26.627232    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 03:48:26.627444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:26.627289    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 03:48:27.007576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:27.007536    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:27.628056 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:27.628020    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 31 03:48:27.628487 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:27.628079    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 31 03:48:28.916419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:28.916387    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:28.917167 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:28.917138    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:28.918057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:28.918039    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 03:48:31.584747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:31.584708    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:36.097729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:36.097694    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:36.381719 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:48:36.381637    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:48:40.605877 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:40.605827    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:45.107947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:45.107908    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:49.602163 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:49.602076    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:51.876715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:51.876611    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"]
Oct 31 03:48:51.887980 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:51.887943    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:51.940539 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:51.940503    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"]
Oct 31 03:48:52.020688 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.020648    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnhx5\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.020892 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.020709    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.020974 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.020948    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.021067 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.021049    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.021106 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.021085    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.021150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.021109    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.021305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.021278    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.021377 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.021325    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.021477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.021456    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.122657 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122623    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.122862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122665    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.122862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122687    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.122862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122764    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.122862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122785    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.122862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122839    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.123146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122901    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnhx5\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.123146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.122944    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.123146 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.123014    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.123363 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.123336    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.123428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.123396    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.123883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.123856    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.124247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.124199    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.124412 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.124262    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.125959 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.125933    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.126053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.125954    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.185638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.185556    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.186229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.186187    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnhx5\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.211922 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.211884    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:52.679737 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.679710    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"]
Oct 31 03:48:52.680782 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.680741    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:48:52.680872 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.680831    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:48:52.680910 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:52.680878    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:48:53.464231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:53.462491    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" event={"ID":"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605","Type":"ContainerStarted","Data":"ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564"}
Oct 31 03:48:53.464231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:53.462559    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" event={"ID":"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605","Type":"ContainerStarted","Data":"e5dbc92659818705082754863a795e2def6b98f12897f9abe78020aa32ed2718"}
Oct 31 03:48:53.639260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:53.639194    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podStartSLOduration=2.639180529 podStartE2EDuration="2.639180529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:48:53.638652551 +0000 UTC m=+1183.472512868" watchObservedRunningTime="2026-10-31 03:48:53.639180529 +0000 UTC m=+1183.473040834"
Oct 31 03:48:54.212402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:54.212365    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:48:54.214046 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:54.214023    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:48:54.214153 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:54.214078    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 31 03:48:54.289694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:54.289658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:55.213015 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:55.212972    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:48:55.213554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:55.213049    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 31 03:48:56.213440 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:56.213392    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:48:56.213952 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:56.213451    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 31 03:48:57.212999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:57.212962    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:48:57.213216 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:57.213038    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 31 03:48:58.212997 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.212946    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:48:58.213506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.213011    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 31 03:48:58.897778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.897742    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"]
Oct 31 03:48:58.901892 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.901868    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.949989 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.949942    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.950187 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.949999    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.950187 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.950154    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.950350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.950238    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ncbd\" (UniqueName: \"kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.950350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.950295    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.950350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.950327    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:58.960799 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.960749    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"]
Oct 31 03:48:58.996008 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:58.995975    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:59.008518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.008486    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"kube-root-ca.crt\""
Oct 31 03:48:59.008518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.008486    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"openshift-service-ca.crt\""
Oct 31 03:48:59.008518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.008486    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"distinct-w5-e5b8f9-kserve-self-signed-certs\""
Oct 31 03:48:59.008518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.008521    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"default-dockercfg-57f6t\""
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.051964    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.052528    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ncbd\" (UniqueName: \"kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.052592    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.052624    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.052718    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.052742    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.052433    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062298 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.059391    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.062831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.062589    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.072630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.062984    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.083058 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.083024    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.138966 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.138930    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ncbd\" (UniqueName: \"kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd\") pod \"distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.140082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.140042    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:48:59.213267 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.213160    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:48:59.213625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.213263    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 31 03:48:59.216416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.216395    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:48:59.687533 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:48:59.687503    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"]
Oct 31 03:48:59.690120 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:48:59.690090    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee59a0e5_9bf0_4541_b1ca_b5c6fa6e784b.slice/crio-624bfd01263521695583e9be7f14cf3d416754e3157fa729526e3ba6db203d06 WatchSource:0}: Error finding container 624bfd01263521695583e9be7f14cf3d416754e3157fa729526e3ba6db203d06: Status 404 returned error can't find the container with id 624bfd01263521695583e9be7f14cf3d416754e3157fa729526e3ba6db203d06
Oct 31 03:49:00.233511 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:00.233481    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:49:00.233954 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:00.233898    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:49:00.234818 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:00.234783    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 03:49:00.529362 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:00.529309    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" event={"ID":"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b","Type":"ContainerStarted","Data":"919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739"}
Oct 31 03:49:00.529701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:00.529668    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" event={"ID":"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b","Type":"ContainerStarted","Data":"624bfd01263521695583e9be7f14cf3d416754e3157fa729526e3ba6db203d06"}
Oct 31 03:49:00.676516 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:00.676453    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podStartSLOduration=2.6764371000000002 podStartE2EDuration="2.6764371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:49:00.676124957 +0000 UTC m=+1190.509985261" watchObservedRunningTime="2026-10-31 03:49:00.6764371 +0000 UTC m=+1190.510297406"
Oct 31 03:49:01.216532 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:01.216499    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:49:03.608292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:03.608253    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:06.381844 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:49:06.381802    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:49:08.430485 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:08.430447    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:09.216828 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:09.216790    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:49:09.230569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:09.230536    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:49:09.633629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:09.633595    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:49:12.997660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:12.997627    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:17.564900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:17.564861    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:20.638444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:20.638406    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:22.001990 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:22.001958    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:26.506999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:26.506965    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:26.903434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:26.903345    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:31.080318 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:31.080277    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:33.691181 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:33.691143    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:35.525128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:35.525094    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:36.384095 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:49:36.384051    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:49:37.592087 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.592053    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"]
Oct 31 03:49:37.596408 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.596386    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.706442 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.706404    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"]
Oct 31 03:49:37.726052 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726014    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726079    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nvd5\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726188    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726258    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726302    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726343    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726368    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726414    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.726491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.726490    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.827817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827783    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nvd5\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827830    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827854    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827871    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827901    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827925    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.827954    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828278    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828292    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828335    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828398    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828635 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828579    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.828761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.828744    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.830264 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.830245    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.830497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.830480    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.890893 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.890814    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.891027 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.890931    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nvd5\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:37.909774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:37.909748    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:38.338736 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:49:38.338699    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86373121_d21a_454f_a4b8_d63a27f273f4.slice/crio-5c018857fadcd5e182fcca0522bf29c43aec7ca0315d908301772cad430a5dfc WatchSource:0}: Error finding container 5c018857fadcd5e182fcca0522bf29c43aec7ca0315d908301772cad430a5dfc: Status 404 returned error can't find the container with id 5c018857fadcd5e182fcca0522bf29c43aec7ca0315d908301772cad430a5dfc
Oct 31 03:49:38.340590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.340565    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"]
Oct 31 03:49:38.341394 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.341359    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:49:38.341482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.341428    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:49:38.341482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.341458    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:49:38.812333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.812295    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" event={"ID":"86373121-d21a-454f-a4b8-d63a27f273f4","Type":"ContainerStarted","Data":"150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30"}
Oct 31 03:49:38.812762 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.812346    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" event={"ID":"86373121-d21a-454f-a4b8-d63a27f273f4","Type":"ContainerStarted","Data":"5c018857fadcd5e182fcca0522bf29c43aec7ca0315d908301772cad430a5dfc"}
Oct 31 03:49:38.910750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.910706    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:38.936701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:38.936644    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podStartSLOduration=1.936623656 podStartE2EDuration="1.936623656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:49:38.934912077 +0000 UTC m=+1228.768772384" watchObservedRunningTime="2026-10-31 03:49:38.936623656 +0000 UTC m=+1228.770483960"
Oct 31 03:49:39.911493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:39.911458    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:49:39.911916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:39.911526    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 03:49:40.017968 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:40.017930    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:40.910837 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:40.910799    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:49:40.911052 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:40.910859    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 03:49:41.910141 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:41.910099    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:49:41.910554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:41.910159    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 03:49:42.910863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:42.910825    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:49:42.911316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:42.910886    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 03:49:43.364843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:43.364803    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:43.910754 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:43.910716    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:49:43.910943 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:43.910787    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 03:49:44.633967 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:44.633927    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:44.910935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:44.910852    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:49:44.910935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:44.910919    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 31 03:49:45.915092 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:45.915060    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:45.915617 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:45.915525    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:45.916242 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:45.916199    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 03:49:49.311659 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:49.311626    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:52.667819 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:52.667768    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:53.693987 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:53.693947    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:49:58.115268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:49:58.115186    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:02.648231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:02.647585    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:02.803663 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:02.803630    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:06.381579 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:50:06.381543    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:50:07.194869 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:07.194836    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:09.653279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.653243    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"]
Oct 31 03:50:09.656873 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.656855    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.713246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.713199    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"]
Oct 31 03:50:09.735313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.735281    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:09.762459 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.762423    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-a64fd2\"/\"openshift-service-ca.crt\""
Oct 31 03:50:09.762643 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.762484    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-a64fd2\"/\"kube-root-ca.crt\""
Oct 31 03:50:09.762643 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.762567    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a64fd2\"/\"distinct-w2-a64fd2-kserve-self-signed-certs\""
Oct 31 03:50:09.764879 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.764860    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a64fd2\"/\"default-dockercfg-xrsfg\""
Oct 31 03:50:09.810268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.809092    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.810268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.809151    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hd6r\" (UniqueName: \"kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.816405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.814466    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.816405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.814637    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.816405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.814730    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.816405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.814773    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.915962    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916057 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916038    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916069    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916133    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916160    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hd6r\" (UniqueName: \"kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916260    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916622 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916592    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916636    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.916792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.916663    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.918606 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.918583    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.918694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.918674    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:09.979671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:09.979628    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hd6r\" (UniqueName: \"kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r\") pod \"distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") " pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:10.323704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:10.323670    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-a64fd2\"/\"default-dockercfg-xrsfg\""
Oct 31 03:50:10.330867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:10.330839    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:10.526342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:10.526313    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:10.669421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:10.669337    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:10.713243 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:50:10.713200    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod169384ae_a553_458b_b56b_7c311ab17d38.slice/crio-710f912d281d777023eab2b537992bfac64a173fdac10eae2122438c4f66c1a6 WatchSource:0}: Error finding container 710f912d281d777023eab2b537992bfac64a173fdac10eae2122438c4f66c1a6: Status 404 returned error can't find the container with id 710f912d281d777023eab2b537992bfac64a173fdac10eae2122438c4f66c1a6
Oct 31 03:50:10.713401 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:10.713360    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"]
Oct 31 03:50:11.081168 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:11.081123    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" event={"ID":"169384ae-a553-458b-b56b-7c311ab17d38","Type":"ContainerStarted","Data":"0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1"}
Oct 31 03:50:11.081168 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:11.081174    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" event={"ID":"169384ae-a553-458b-b56b-7c311ab17d38","Type":"ContainerStarted","Data":"710f912d281d777023eab2b537992bfac64a173fdac10eae2122438c4f66c1a6"}
Oct 31 03:50:11.201717 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:11.201655    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podStartSLOduration=2.20164264 podStartE2EDuration="2.20164264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:50:11.200964848 +0000 UTC m=+1261.034825152" watchObservedRunningTime="2026-10-31 03:50:11.20164264 +0000 UTC m=+1261.035502945"
Oct 31 03:50:11.332225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:11.332121    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:11.702131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:11.702049    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:16.212699 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:16.212659    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:20.331901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:20.331806    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:20.352995 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:20.352967    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:20.698772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:20.698669    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:21.003227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:20.997844    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:21.151141 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:21.151115    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:50:25.287090 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:25.287051    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:29.798846 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:29.798811    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:34.305356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:34.305324    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:36.382167 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:50:36.382128    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:50:37.659481 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.659439    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"]
Oct 31 03:50:37.664201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.664176    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.773227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.773171    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"]
Oct 31 03:50:37.802829 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.802793    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803002 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.802835    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803002 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.802927    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5tdh\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.803026    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803110 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.803094    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803155 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.803132    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.803193    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803270 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.803232    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.803346 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.803332    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905166    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905239    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905279    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5tdh\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905357    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905457    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905480    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905512    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905576 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905545    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.905739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.905633    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.906485 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.906436    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.906630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.906502    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.906630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.906548    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.906630 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.906584    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.907176 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.907156    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.908919 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.908893    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.909013 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.908991    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.968672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.968595    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.968806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.968717    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5tdh\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:37.979566 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:37.979541    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:38.406258 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:50:38.406192    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf7a4364_e1d3_4f50_beb7_3a0c88346fd9.slice/crio-b0cb30dbf3172564561cff6be13d90fdf4704059dfcafae1980e05395a92063e WatchSource:0}: Error finding container b0cb30dbf3172564561cff6be13d90fdf4704059dfcafae1980e05395a92063e: Status 404 returned error can't find the container with id b0cb30dbf3172564561cff6be13d90fdf4704059dfcafae1980e05395a92063e
Oct 31 03:50:38.408749 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:38.408716    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:50:38.408831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:38.408788    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:50:38.408831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:38.408817    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:50:38.409140 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:38.409120    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"]
Oct 31 03:50:38.810015 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:38.809965    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:39.253826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:39.253777    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" event={"ID":"af7a4364-e1d3-4f50-beb7-3a0c88346fd9","Type":"ContainerStarted","Data":"b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586"}
Oct 31 03:50:39.254230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:39.254104    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" event={"ID":"af7a4364-e1d3-4f50-beb7-3a0c88346fd9","Type":"ContainerStarted","Data":"b0cb30dbf3172564561cff6be13d90fdf4704059dfcafae1980e05395a92063e"}
Oct 31 03:50:39.433746 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:39.433692    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podStartSLOduration=2.433675684 podStartE2EDuration="2.433675684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:50:39.431071611 +0000 UTC m=+1289.264931927" watchObservedRunningTime="2026-10-31 03:50:39.433675684 +0000 UTC m=+1289.267535989"
Oct 31 03:50:39.979766 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:39.979723    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:39.981288 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:39.981258    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:50:39.981402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:39.981315    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 03:50:40.980794 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:40.980758    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:50:40.981260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:40.980816    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 03:50:41.980325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:41.980287    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:50:41.980529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:41.980346    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 03:50:42.980032 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:42.979996    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:50:42.980548 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:42.980056    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 03:50:43.366931 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:43.366889    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:43.980755 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:43.980714    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:50:43.981282 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:43.980787    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 03:50:44.980476 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:44.980439    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:50:44.980665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:44.980499    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 31 03:50:45.986073 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:45.986040    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:45.986505 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:45.986482    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:45.987166 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:45.987148    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 03:50:47.916244 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:47.916190    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:52.387591 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:52.387555    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:56.905940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:56.905904    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:50:57.046443 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:50:57.046412    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:01.412702 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:01.412667    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:05.897564 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:05.897525    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:06.382281 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:51:06.382244    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:51:10.384631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:10.384600    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:14.926426 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:14.926392    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:19.398037 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:19.397999    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:21.314923 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:21.314888    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:23.969395 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:23.969355    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:27.135806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:27.135762    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:28.635258 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:28.635220    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:29.845665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:29.845633    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:33.072863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:33.072800    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:36.381556 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:51:36.381514    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:51:37.562172 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:37.562134    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:42.125195 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:42.125161    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:45.080455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:45.080422    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:45.923042 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:45.923005    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:46.611749 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:46.611716    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:52.383827 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:52.383735    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:55.645430 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:55.645391    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:56.014004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:56.013974    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:56.154237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:56.154181    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:51:56.637671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:51:56.637636    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:00.065097 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:00.065054    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:00.214066 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:00.213291    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:04.592023 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:04.591977    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:06.431262 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:52:06.431199    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:52:08.012285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.012093    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"]
Oct 31 03:52:08.039248 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.039183    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:08.111514 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.111469    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"]
Oct 31 03:52:08.111831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.111807    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.212773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.212743    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.212773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.212776    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd98s\" (UniqueName: \"kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.212985 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.212901    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.212985 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.212964    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.213108 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.213084    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.213148 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.213137    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.280054 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.279970    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"kube-root-ca.crt\""
Oct 31 03:52:08.280250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.279967    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"openshift-service-ca.crt\""
Oct 31 03:52:08.280250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.280011    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"default-dockercfg-kh5kd\""
Oct 31 03:52:08.280250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.280128    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"distinct-2-w0-ae8e27-kserve-self-signed-certs\""
Oct 31 03:52:08.323879 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.323840    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.324147 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.324130    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.324513 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.324310    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.324690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.324670    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.324960 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.324937    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.325103 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.325087    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd98s\" (UniqueName: \"kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.330839 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.330808    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.331078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.331044    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.331557 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.331531    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.361416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.361359    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.362540 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.362505    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.388968 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.388932    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd98s\" (UniqueName: \"kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s\") pod \"distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.506801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.506766    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:08.884242 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:52:08.884170    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f91b7ef_f035_4f81_951d_9dc17c538592.slice/crio-5d9abc3c9d9a1238c56fad8b075461be5407410f511b98d18346b67c77ec2ebe WatchSource:0}: Error finding container 5d9abc3c9d9a1238c56fad8b075461be5407410f511b98d18346b67c77ec2ebe: Status 404 returned error can't find the container with id 5d9abc3c9d9a1238c56fad8b075461be5407410f511b98d18346b67c77ec2ebe
Oct 31 03:52:08.886831 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.886758    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"]
Oct 31 03:52:08.911474 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:08.911439    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" event={"ID":"7f91b7ef-f035-4f81-951d-9dc17c538592","Type":"ContainerStarted","Data":"5d9abc3c9d9a1238c56fad8b075461be5407410f511b98d18346b67c77ec2ebe"}
Oct 31 03:52:09.330904 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:09.330869    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:09.918705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:09.918663    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" event={"ID":"7f91b7ef-f035-4f81-951d-9dc17c538592","Type":"ContainerStarted","Data":"be9f68d3664ec98ab639a4bf1fb577a5cc8ca157587d5ec98fc2a50c09539111"}
Oct 31 03:52:10.040076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:10.039899    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podStartSLOduration=3.039881857 podStartE2EDuration="3.039881857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:10.038855624 +0000 UTC m=+1379.872715941" watchObservedRunningTime="2026-10-31 03:52:10.039881857 +0000 UTC m=+1379.873742163"
Oct 31 03:52:10.507013 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:10.506983    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:10.902613 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:10.902519    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:11.756547 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:11.756507    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:13.897215 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:13.897184    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:18.442647 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:18.442590    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:18.521263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:18.521121    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:18.538454 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:18.538427    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:18.999848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:18.999818    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:52:21.655651 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:21.655608    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:22.890940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:22.890906    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:23.432493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.432458    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"]
Oct 31 03:52:23.437102 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.437076    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511399    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511539    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511583    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511599    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511765 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511738    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511809 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511783    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511804    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511911 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511891    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.511949 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.511926    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mw5z\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.545696 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.545665    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"]
Oct 31 03:52:23.612947 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.612912    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.612956    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.612977    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613009    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613034    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mw5z\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613140    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613279    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613383    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613451    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613520    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613537    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613726 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613699    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.613783 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613727    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.614004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.613972    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.615523 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.615503    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.615649 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.615633    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.676597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.676563    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.676597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.676576    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mw5z\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:23.751383 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:23.751351    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:24.176007 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:52:24.175974    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f7f3c8_5d5c_4982_aa65_b073260cc330.slice/crio-e15b6731e41e14d2c3aa1c19645318b0aec4e7d416798402f58f68116f0107a1 WatchSource:0}: Error finding container e15b6731e41e14d2c3aa1c19645318b0aec4e7d416798402f58f68116f0107a1: Status 404 returned error can't find the container with id e15b6731e41e14d2c3aa1c19645318b0aec4e7d416798402f58f68116f0107a1
Oct 31 03:52:24.178295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:24.178269    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"]
Oct 31 03:52:24.178856 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:24.178826    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:24.178910 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:24.178892    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:24.178954 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:24.178943    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:25.024828 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:25.024789    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" event={"ID":"f1f7f3c8-5d5c-4982-aa65-b073260cc330","Type":"ContainerStarted","Data":"2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab"}
Oct 31 03:52:25.025014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:25.024853    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" event={"ID":"f1f7f3c8-5d5c-4982-aa65-b073260cc330","Type":"ContainerStarted","Data":"e15b6731e41e14d2c3aa1c19645318b0aec4e7d416798402f58f68116f0107a1"}
Oct 31 03:52:25.166250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:25.154520    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podStartSLOduration=2.154499119 podStartE2EDuration="2.154499119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:25.151706329 +0000 UTC m=+1394.985566632" watchObservedRunningTime="2026-10-31 03:52:25.154499119 +0000 UTC m=+1394.988359425"
Oct 31 03:52:25.760232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:25.760054    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:25.760752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:25.760279    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:52:25.760752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:25.760342    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 03:52:26.766409 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:26.766370    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:52:26.766908 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:26.766440    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 03:52:27.395276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:27.395242    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:27.767757 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:27.767707    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:52:27.768267 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:27.767773    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 03:52:28.766376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:28.766335    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:52:28.766570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:28.766396    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 03:52:29.760730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:29.760690    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:52:29.761385 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:29.761347    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 03:52:30.017688 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:30.017602    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:30.752695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:30.752654    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:52:30.752927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:30.752729    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 31 03:52:31.228358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.228261    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"]
Oct 31 03:52:31.233343 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.233317    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.287627 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.287594    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-d7b7bc-openshift-default-dockercfg-nszpp\""
Oct 31 03:52:31.343631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.343589    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"]
Oct 31 03:52:31.347268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347200    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347330    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347507 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347484    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347534    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347690    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347724    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347890 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347852    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8nkj\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.347966 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.347948    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.348046 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.348029    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449099    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449153    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449244    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449292    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449535 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449386    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449535 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449421    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449560    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449581    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449638 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449590    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449798 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449649    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449798 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449709    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8nkj\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.449908 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.449881    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.450141 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.450117    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.450252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.450223    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.452109 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.452083    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.452553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.452532    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.512814 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.512784    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.512980 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.512859    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8nkj\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.548137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.548102    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:31.757397 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.757353    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:31.757728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.757705    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:31.758568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.758552    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 03:52:31.975370 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:52:31.975329    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025dfe4c_2482_43cf_bdac_a8e6f273a556.slice/crio-ae54173c9305924f76243e8f6929c99fd4a04aae106aa96be093e6cf16846bb3 WatchSource:0}: Error finding container ae54173c9305924f76243e8f6929c99fd4a04aae106aa96be093e6cf16846bb3: Status 404 returned error can't find the container with id ae54173c9305924f76243e8f6929c99fd4a04aae106aa96be093e6cf16846bb3
Oct 31 03:52:31.976926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.976889    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"]
Oct 31 03:52:31.980256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.978623    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:31.980256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.978783    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:31.980256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.978893    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:31.992158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:31.992130    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:32.105704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:32.105661    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" event={"ID":"025dfe4c-2482-43cf-bdac-a8e6f273a556","Type":"ContainerStarted","Data":"88a26dc39cd18dc9c0b2b0c0bd748831b723d665f284a38c1fd78a96d0aa34cf"}
Oct 31 03:52:32.105894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:32.105767    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" event={"ID":"025dfe4c-2482-43cf-bdac-a8e6f273a556","Type":"ContainerStarted","Data":"ae54173c9305924f76243e8f6929c99fd4a04aae106aa96be093e6cf16846bb3"}
Oct 31 03:52:32.235909 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:32.235810    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podStartSLOduration=1.235792714 podStartE2EDuration="1.235792714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:32.234311507 +0000 UTC m=+1402.068171812" watchObservedRunningTime="2026-10-31 03:52:32.235792714 +0000 UTC m=+1402.069653020"
Oct 31 03:52:32.549496 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:32.549461    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:33.553354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:33.553314    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:33.553984 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:33.553947    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:34.557626 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:34.557587    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:34.559018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:34.558979    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:35.553229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:35.552644    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:35.553229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:35.552714    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:36.422667 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:52:36.422631    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:52:36.551652 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:36.551609    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:36.551953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:36.551924    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:36.911990 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:36.911955    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:37.558230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:37.557618    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:37.558230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:37.557683    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:38.555243 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:38.554640    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:38.555243 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:38.554709    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:39.549565 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:39.549530    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:52:39.549956 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:39.549584    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 31 03:52:40.631247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:40.631169    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:40.631760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:40.631710    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:40.633587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:40.633564    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:52:41.506744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:41.506708    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:45.995959 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:45.995919    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:50.609905 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:50.609869    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:53.034305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.034262    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"]
Oct 31 03:52:53.040812 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.040786    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.071924 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.071875    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.071990    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072037    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072065    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbgcp\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072137 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072125    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072451 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072156    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072451 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072243    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072451 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072305    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.072628 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.072522    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.114523 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.114480    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"]
Oct 31 03:52:53.174595 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.174557    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.174806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.174651    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.174806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.174682    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175365 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175338    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175561 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175532    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175648 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175075    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175648 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175543    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175660    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175694    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbgcp\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175770    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.175861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.175807    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.176053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.176024    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.176222 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.176185    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.176312 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.176300    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.177934 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.177908    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.178173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.178154    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.239171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.239131    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.239399 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.239371    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbgcp\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.359675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.359585    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:53.784062 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:52:53.784030    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79d3196_a04c_40a0_9a08_f1cb72a4a870.slice/crio-135e4d8c14c388d571530f867a5ef3f33555f9a58efcbda33aa6cc26f4fc6fcf WatchSource:0}: Error finding container 135e4d8c14c388d571530f867a5ef3f33555f9a58efcbda33aa6cc26f4fc6fcf: Status 404 returned error can't find the container with id 135e4d8c14c388d571530f867a5ef3f33555f9a58efcbda33aa6cc26f4fc6fcf
Oct 31 03:52:53.786635 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.786606    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"]
Oct 31 03:52:53.787163 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.787125    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:53.787251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.787218    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:53.787293 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:53.787255    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:52:54.330620 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:54.330582    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" event={"ID":"a79d3196-a04c-40a0-9a08-f1cb72a4a870","Type":"ContainerStarted","Data":"4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89"}
Oct 31 03:52:54.331047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:54.330652    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" event={"ID":"a79d3196-a04c-40a0-9a08-f1cb72a4a870","Type":"ContainerStarted","Data":"135e4d8c14c388d571530f867a5ef3f33555f9a58efcbda33aa6cc26f4fc6fcf"}
Oct 31 03:52:54.363235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:54.362570    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:52:54.363235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:54.362637    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:52:54.374687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:54.374654    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:52:54.547140 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:54.547073    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podStartSLOduration=2.547051444 podStartE2EDuration="2.547051444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:54.546621704 +0000 UTC m=+1424.380482027" watchObservedRunningTime="2026-10-31 03:52:54.547051444 +0000 UTC m=+1424.380911750"
Oct 31 03:52:55.361228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:55.360290    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:52:55.361228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:55.360352    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:52:55.730863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:55.730781    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:52:56.367182 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:56.367140    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:52:56.367854 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:56.367812    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:52:57.361657 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:57.361619    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:52:57.361957 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:57.361928    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:52:58.366432 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:58.366398    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:52:58.366918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:58.366883    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:52:59.366735 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:59.366691    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:52:59.367161 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:52:59.366770    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:53:00.360742 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:00.360702    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:53:00.360917 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:00.360770    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 31 03:53:00.841699 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:00.841663    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:01.371065 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:01.371024    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:53:01.371381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:01.371347    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:53:01.372017 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:01.372000    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 03:53:04.716073 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:04.716037    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:04.858564 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:04.858530    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:05.437686 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:05.437651    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:06.383033 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:53:06.382985    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:53:07.731784 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.731748    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"]
Oct 31 03:53:07.736077 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.736057    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.833923 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.833882    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"]
Oct 31 03:53:07.901666 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901621    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.901878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901715    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.901878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901810    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.901878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901841    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.901878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901857    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.902036 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901949    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.902036 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.901978    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlk88\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.902128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.902110    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:07.902300 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:07.902282    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004025 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.003990    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004025 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004029    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004052    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004167    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004195    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004295    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004332    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlk88\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004408    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004589 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004548    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004589 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004556    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004584    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004820    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004982 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004910    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.004982 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.004975    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.006568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.006543    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.007022 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.007001    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.067254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.067198    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.067409 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.067348    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlk88\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.146505 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.146461    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"]
Oct 31 03:53:08.154049 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.154015    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.207825 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.207789    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"]
Oct 31 03:53:08.308380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308284    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308354    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308442    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308486    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308658 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308594    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308687    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308743 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308732    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308807    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.308855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.308826    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzl7b\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.350477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.350447    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:08.410742 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.410713    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.410826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.410775    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.410863 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.410823    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.410896 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.410876    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.410916    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.410939    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzl7b\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.411003    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.411045    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.411091    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.411818    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.411847    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.411958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.411829    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.412093 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.412069    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.412771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.412744    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.415533 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.415474    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.421228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.418270    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.474794 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.474752    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.474940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.474819    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzl7b\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.769308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.769276    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:08.780801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.780774    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"]
Oct 31 03:53:08.784256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.784217    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:08.784356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.784302    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:08.784356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:08.784345    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:09.199809 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:53:09.198945    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00104874_4cc9_4129_8923_0e301106d24e.slice/crio-7571d085d856656b461aec7e9ac6b9d45428f4964483dea7df1e50a2bf8ec1e8 WatchSource:0}: Error finding container 7571d085d856656b461aec7e9ac6b9d45428f4964483dea7df1e50a2bf8ec1e8: Status 404 returned error can't find the container with id 7571d085d856656b461aec7e9ac6b9d45428f4964483dea7df1e50a2bf8ec1e8
Oct 31 03:53:09.201302 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.201219    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"]
Oct 31 03:53:09.201953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.201921    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:09.202037 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.201985    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:09.202037 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.202027    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:09.351356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.351320    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"]
Oct 31 03:53:09.355541 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.355517    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.439991 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.439151    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"]
Oct 31 03:53:09.451324 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.451239    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" event={"ID":"6f04960d-4eff-4aeb-adfa-3f547833d2ee","Type":"ContainerStarted","Data":"47cfd9ed91d68864bb2ba1d8e687cdc3a249a165df75b8a02462692093b13d26"}
Oct 31 03:53:09.451563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.451539    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" event={"ID":"6f04960d-4eff-4aeb-adfa-3f547833d2ee","Type":"ContainerStarted","Data":"af0c3a5ea9fbeadc425bff7f004ad7f6d51f20b998dbc422bed39c3882544ac6"}
Oct 31 03:53:09.454644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.454615    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" event={"ID":"00104874-4cc9-4129-8923-0e301106d24e","Type":"ContainerStarted","Data":"0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd"}
Oct 31 03:53:09.454891 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.454663    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" event={"ID":"00104874-4cc9-4129-8923-0e301106d24e","Type":"ContainerStarted","Data":"7571d085d856656b461aec7e9ac6b9d45428f4964483dea7df1e50a2bf8ec1e8"}
Oct 31 03:53:09.527956 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.527915    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.527979    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrhp\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528080    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528124    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528147    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528441 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528407    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528575 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528526    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528833 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528792    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.528956 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.528929    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.601413 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.601356    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podStartSLOduration=1.6013381249999998 podStartE2EDuration="1.601338125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:09.599287455 +0000 UTC m=+1439.433147759" watchObservedRunningTime="2026-10-31 03:53:09.601338125 +0000 UTC m=+1439.435198430"
Oct 31 03:53:09.629713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.629672    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.629919 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.629730    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630011 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.629928    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630011    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630061    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630112    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrhp\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630143    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630170    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630195    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630556 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630528    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.630912 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630877    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.631045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.630885    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.631433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.631407    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.631880 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.631859    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.632909 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.632884    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.633213 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.633181    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.693891 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.693849    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.694088 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.694065    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrhp\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:09.727132 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.727010    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podStartSLOduration=2.726993966 podStartE2EDuration="2.726993966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:09.724337513 +0000 UTC m=+1439.558197815" watchObservedRunningTime="2026-10-31 03:53:09.726993966 +0000 UTC m=+1439.560854271"
Oct 31 03:53:09.769499 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.769457    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:09.971846 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:09.971475    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:10.022768 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.022694    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:10.403752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.403718    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:10.404103 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.404069    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:10.408680 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.408615    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:10.597629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.591333    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"]
Oct 31 03:53:10.600457 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:53:10.600424    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0300d288_3aec_4909_89c8_45f8ac68cbbd.slice/crio-aca8be811513f8a2f36fff39b55592c000e85c44778beec86d7f4518436a859c WatchSource:0}: Error finding container aca8be811513f8a2f36fff39b55592c000e85c44778beec86d7f4518436a859c: Status 404 returned error can't find the container with id aca8be811513f8a2f36fff39b55592c000e85c44778beec86d7f4518436a859c
Oct 31 03:53:10.603481 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.603439    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:10.603586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.603533    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:10.603586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.603572    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:10.773230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.772343    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:10.773230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:10.772406    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:11.371228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.354649    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:11.371228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.354710    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:11.570230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.569952    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" event={"ID":"0300d288-3aec-4909-89c8-45f8ac68cbbd","Type":"ContainerStarted","Data":"1747f294336bfb4fc6acc8e08002098f213cb5c7eaba5bb275024ac8bdd4a7fb"}
Oct 31 03:53:11.570230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.570066    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" event={"ID":"0300d288-3aec-4909-89c8-45f8ac68cbbd","Type":"ContainerStarted","Data":"aca8be811513f8a2f36fff39b55592c000e85c44778beec86d7f4518436a859c"}
Oct 31 03:53:11.709292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.709053    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podStartSLOduration=2.7090321 podStartE2EDuration="2.7090321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:11.698736823 +0000 UTC m=+1441.532597129" watchObservedRunningTime="2026-10-31 03:53:11.7090321 +0000 UTC m=+1441.542892406"
Oct 31 03:53:11.773241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.772714    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:11.773241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.772778    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:12.009301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.975021    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:12.009301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.975225    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:12.009301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:11.975271    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:12.384297 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:12.383438    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:12.384297 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:12.383564    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:12.771132 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:12.770624    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:12.771132 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:12.770692    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:13.021793 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.021615    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.022396 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.021702    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:13.351383 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.351270    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.351657 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.351614    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:13.636644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.635969    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 31 03:53:13.776229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.775634    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.776229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.775700    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:13.976287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.975665    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.976287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:13.975730    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:14.030232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.030132    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.144232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.141458    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.373265 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.361228    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:14.373265 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.361294    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:14.684127 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.682380    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.718231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.714366    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:53:14.762248 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.726629    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:53:14.762248 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.726699    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.797372 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.797200    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:14.797372 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.797284    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:14.852052 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:14.852008    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:15.141641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.121687    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 03:53:15.146230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.142988    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:53:15.221272 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.167559    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.270249 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.248541    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:53:15.320373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.318716    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:15.320373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.318780    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:15.368288 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.368251    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:15.368557 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.368527    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:15.503321 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.494311    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:53:15.715232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.684409    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.890103 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.889799    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:15.894339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:15.890351    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:16.006626 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.006588    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:16.104232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.100879    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:16.322752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.322697    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 31 03:53:16.447251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.445416    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:16.447251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.445494    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:16.525264 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.517494    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:16.790356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.783479    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:16.790356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.783545    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:16.791473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.791433    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:53:16.828867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:16.828833    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:53:17.027917 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.027879    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.028177 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.028149    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:17.380231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.379219    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.380231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.379284    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:17.785793 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.782646    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.785793 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.782716    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 31 03:53:17.972815 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.972759    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.973128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:17.973098    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:18.082387 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.082297    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"]
Oct 31 03:53:18.085024 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.085001    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4885f733-k4stl"]
Oct 31 03:53:18.352060 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.351963    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:53:18.371348 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.371295    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 31 03:53:18.393901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.393868    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b66e2fa9-8f1a-45d8-ace5-0b54b42ad514" path="/var/lib/kubelet/pods/b66e2fa9-8f1a-45d8-ace5-0b54b42ad514/volumes"
Oct 31 03:53:18.977711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.977669    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:18.978014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:18.977982    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:19.310314 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.310278    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:19.311471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.311447    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:19.312480 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.312461    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 03:53:19.355425 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.355389    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:19.355748 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.355726    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:19.356569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.356549    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:53:19.512559 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.512525    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:19.973085 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.973050    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:53:19.973320 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:19.973125    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 31 03:53:20.978538 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:20.978508    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:20.979007 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:20.978864    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:20.979781 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:20.979758    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 03:53:21.999973 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:21.999933    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:23.999433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:23.999389    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:24.453512 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.453426    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"]
Oct 31 03:53:24.459194 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.459168    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494366 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494333    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494405    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494513    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbltq\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494563 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494553    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494705    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494738    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494871 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494847    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494925 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494890    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.494925 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.494915    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.559886 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.559853    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"]
Oct 31 03:53:24.596154 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596112    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596232    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596406 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596347    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596424    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596478    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbltq\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596503    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596573    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596601    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.596739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.596667    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.597337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.597104    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.597337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.597120    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.597337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.597138    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.597337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.597134    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.597560 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.597352    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.599456 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.599435    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.599653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.599633    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.660062 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.660029    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.660236 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.660110    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbltq\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:24.773481 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:24.773444    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:25.200222 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:53:25.200176    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc248aba_1443_49ca_a0da_4ec62f4f0828.slice/crio-f75623e170fc23cc0c2f85387e4da895061c6e0f0d709b1c019b7365f9c63e12 WatchSource:0}: Error finding container f75623e170fc23cc0c2f85387e4da895061c6e0f0d709b1c019b7365f9c63e12: Status 404 returned error can't find the container with id f75623e170fc23cc0c2f85387e4da895061c6e0f0d709b1c019b7365f9c63e12
Oct 31 03:53:25.203053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:25.203004    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:25.203174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:25.203101    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:25.203174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:25.203150    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:53:25.203362 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:25.203341    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"]
Oct 31 03:53:26.037188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:26.037152    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" event={"ID":"dc248aba-1443-49ca-a0da-4ec62f4f0828","Type":"ContainerStarted","Data":"3e1e754575b36de3234b20dfcde4ad406a12ab23617d2569760bb2ee6c6a5672"}
Oct 31 03:53:26.037393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:26.037250    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" event={"ID":"dc248aba-1443-49ca-a0da-4ec62f4f0828","Type":"ContainerStarted","Data":"f75623e170fc23cc0c2f85387e4da895061c6e0f0d709b1c019b7365f9c63e12"}
Oct 31 03:53:26.224117 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:26.224067    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podStartSLOduration=2.224052319 podStartE2EDuration="2.224052319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:26.223181703 +0000 UTC m=+1456.057042038" watchObservedRunningTime="2026-10-31 03:53:26.224052319 +0000 UTC m=+1456.057912624"
Oct 31 03:53:26.773654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:26.773606    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:26.775138 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:26.775108    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 03:53:26.775291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:26.775161    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 03:53:27.774185 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:27.774149    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 03:53:27.774636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:27.774221    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 03:53:28.626671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:28.626634    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:28.783297 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:28.782793    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 03:53:28.783297 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:28.782854    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 03:53:29.774517 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:29.774484    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 03:53:29.774703 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:29.774538    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 03:53:30.774600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:30.774560    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 03:53:30.774985 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:30.774614    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 03:53:31.774429 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:31.774391    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 03:53:31.774681 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:31.774485    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 03:53:32.778546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:32.778515    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:32.778977 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:32.778952    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:32.779745 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:32.779723    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 03:53:33.123695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:33.123658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:36.383990 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:53:36.383948    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:53:37.724605 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:37.724569    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:42.292508 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:42.292451    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:44.767375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:44.767326    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:47.402579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:47.402534    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:51.919427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:51.919384    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:53:56.500591 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:53:56.500554    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:01.015783 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:01.015735    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:05.486179 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:05.486142    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:06.382826 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:54:06.382791    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:54:08.202008 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.201970    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"]
Oct 31 03:54:08.206554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.206532    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.289944 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.289910    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"]
Oct 31 03:54:08.359932 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.359899    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360020    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360171 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360148    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360304 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360280    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwpqx\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360393 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360364    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360462 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360399    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360509 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360466    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360557 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360525    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.360612 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.360601    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.461998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.461918    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.461998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.461979    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwpqx\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462031    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462058    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462073    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462194    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462369    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462398    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462462    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462484    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462557    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462715 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462705    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.462828 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.462803    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.463183 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.463159    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.464494 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.464474    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.464706 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.464689    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.524771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.524730    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.524938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.524918    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwpqx\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:08.821876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:08.821843    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:09.248741 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:54:09.248705    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bfddfc3_1b1a_4166_b8da_181cfd6538cb.slice/crio-f154dfd430b44480e36ef54ad9b8553c3198e15fb3377973429bbfe5055d1499 WatchSource:0}: Error finding container f154dfd430b44480e36ef54ad9b8553c3198e15fb3377973429bbfe5055d1499: Status 404 returned error can't find the container with id f154dfd430b44480e36ef54ad9b8553c3198e15fb3377973429bbfe5055d1499
Oct 31 03:54:09.250854 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.250813    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"]
Oct 31 03:54:09.251531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.251503    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:09.251589 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.251561    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:09.251636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.251591    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:09.396983 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.396950    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" event={"ID":"8bfddfc3-1b1a-4166-b8da-181cfd6538cb","Type":"ContainerStarted","Data":"f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6"}
Oct 31 03:54:09.397165 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.397005    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" event={"ID":"8bfddfc3-1b1a-4166-b8da-181cfd6538cb","Type":"ContainerStarted","Data":"f154dfd430b44480e36ef54ad9b8553c3198e15fb3377973429bbfe5055d1499"}
Oct 31 03:54:09.822817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:09.822773    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:10.059365 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:10.059302    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:10.824303 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:10.824270    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:54:10.824711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:10.824329    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 31 03:54:11.822588 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:11.822552    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:54:11.822758 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:11.822625    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 31 03:54:12.822440 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:12.822400    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:54:12.822832 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:12.822459    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 31 03:54:13.822234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:13.822184    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:54:13.822412 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:13.822254    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 31 03:54:14.689969 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:14.689904    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:14.824555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:14.824045    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:54:14.824555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:14.824145    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 31 03:54:15.823000 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:15.822952    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:54:15.823500 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:15.823025    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 31 03:54:17.023820 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.022548    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:17.024942 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.024918    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:17.026272 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.026249    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 03:54:17.058806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.058751    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podStartSLOduration=9.058737431 podStartE2EDuration="9.058737431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:09.527634915 +0000 UTC m=+1499.361495221" watchObservedRunningTime="2026-10-31 03:54:17.058737431 +0000 UTC m=+1506.892597745"
Oct 31 03:54:17.061777 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.061746    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"]
Oct 31 03:54:17.066387 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.066366    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214242    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214439    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214458    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"]
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214515    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214561    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214615    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214661    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214752    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh4ls\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.214780    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.215739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.215691    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.469290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.348833    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.500375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.357068    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.500728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.500704    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.501014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.500997    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.501186 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.501161    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.501392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.501366    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.501669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.501654    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.501819 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.501803    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.502020 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.502005    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh4ls\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.502135 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.502121    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.502798 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.502776    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.518286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.503858    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.518286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.504463    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.518286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.504943    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.540588 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.540510    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.640503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.610969    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.753290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.727166    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:17.796562 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:17.796528    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh4ls\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:18.039276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:18.007634    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:19.517149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.515448    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.517149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.515798    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 03:54:19.661328 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.652665    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"]
Oct 31 03:54:19.722355 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.722307    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.758123 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.758064    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:19.758571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.758531    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:19.773854 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.765016    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:19.783599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.783508    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" event={"ID":"1a7799e3-3549-488c-becd-5d4d29d98874","Type":"ContainerStarted","Data":"3542c42d02e0ac6e1f4b4db94700be182ab207652f0337ddadb194cc02244aaa"}
Oct 31 03:54:19.836455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:19.836410    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:20.518683 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.516061    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 03:54:20.519546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.519401    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:54:20.519546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.519463    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:20.727243 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.726073    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:20.752592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.741343    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:54:20.752592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.741464    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:54:20.856003 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.855963    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:20.856879 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:20.856848    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" event={"ID":"1a7799e3-3549-488c-becd-5d4d29d98874","Type":"ContainerStarted","Data":"d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e"}
Oct 31 03:54:21.079791 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.079300    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:21.079791 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.079446    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:21.079791 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.079492    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:21.351284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.345361    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": context deadline exceeded"
Oct 31 03:54:21.353876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.353847    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:54:21.480675 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.480608    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podStartSLOduration=4.480585904 podStartE2EDuration="4.480585904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:21.395964779 +0000 UTC m=+1511.229825082" watchObservedRunningTime="2026-10-31 03:54:21.480585904 +0000 UTC m=+1511.314446213"
Oct 31 03:54:21.632978 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:21.632889    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:54:22.034455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:22.034419    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:22.034654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:22.034480    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:22.371654 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:22.371547    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:23.034035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:23.033997    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:23.034571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:23.034060    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:24.034755 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:24.034718    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:24.035181 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:24.034773    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:25.034503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:25.034464    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:25.034691 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:25.034528    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:25.109915 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:25.109875    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:26.034049 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:26.034013    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:26.034246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:26.034064    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:27.034572 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:27.034527    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:27.036879 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:27.034582    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:28.034575 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:28.034536    5906 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:54:28.034983 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:28.034605    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 31 03:54:29.039020 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:29.038982    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:29.039711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:29.039682    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:29.040761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:29.040739    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:54:29.797111 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:29.797073    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:34.281898 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:34.281862    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:36.382785 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:54:36.382746    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:54:38.822064 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:38.822029    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:39.026937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.026899    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:39.062422 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.062382    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"]
Oct 31 03:54:39.067009 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.066990    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.153952 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.153911    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"]
Oct 31 03:54:39.175801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.175767    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.175962    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176100 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176021    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:39.176100 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176055    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176249    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176294    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d9hr\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176372    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176416    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176477 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176454    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.176581 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.176493    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.277609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277567    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.277809 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277657    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.277809 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277708    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.277809 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277743    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d9hr\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.277809 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277783    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277815    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277859    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277884    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.277941    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278330 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.278199    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278330 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.278242    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278551 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.278506    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.278706    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.278900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.278880    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.280452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.280432    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.280717 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.280696    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.353432 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.353363    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.353593 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.353553    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d9hr\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.381633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.381604    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:39.812804 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:54:39.812773    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e707f40_eb62_40b1_a9da_f302f66c83ee.slice/crio-b5e109bbaf166ee026d429f9ffd926849e5c6b16be75c053691bb9c338acb1e9 WatchSource:0}: Error finding container b5e109bbaf166ee026d429f9ffd926849e5c6b16be75c053691bb9c338acb1e9: Status 404 returned error can't find the container with id b5e109bbaf166ee026d429f9ffd926849e5c6b16be75c053691bb9c338acb1e9
Oct 31 03:54:39.816150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.816111    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:39.816281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.816184    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:39.816281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.816251    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:39.819120 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.819094    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"]
Oct 31 03:54:39.841864 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.841836    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"]
Oct 31 03:54:39.847414 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.847393    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887041    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887282 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887124    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887282 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887186    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887282 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887254    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmtdv\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887472 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887290    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887472 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887327    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887472 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887390    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887472 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887453    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.887686 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.887534    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.889834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.889806    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:39.926753 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.926696    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"]
Oct 31 03:54:39.988965 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.988923    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989166 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989009    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989166 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989108    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989176    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989248    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989303    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmtdv\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989345    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989384    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989454    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989497    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989781 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989660    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989896 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989870    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.989999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989976    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.990061 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.989979    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.992126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.992104    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.992263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.992127    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:39.995587 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.995560    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" event={"ID":"2e707f40-eb62-40b1-a9da-f302f66c83ee","Type":"ContainerStarted","Data":"a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd"}
Oct 31 03:54:39.995697 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:39.995612    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" event={"ID":"2e707f40-eb62-40b1-a9da-f302f66c83ee","Type":"ContainerStarted","Data":"b5e109bbaf166ee026d429f9ffd926849e5c6b16be75c053691bb9c338acb1e9"}
Oct 31 03:54:40.027108 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.027072    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"]
Oct 31 03:54:40.032577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.032549    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.052585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.052554    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:40.053920 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.053896    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmtdv\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.090356    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"]
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.091312    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.091349    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.091433    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.091512    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62h4f\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.091906    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.091948    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.092023    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.092177    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.096306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.092753    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.130290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.130228    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podStartSLOduration=1.130196539 podStartE2EDuration="1.130196539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:40.128971535 +0000 UTC m=+1529.962831841" watchObservedRunningTime="2026-10-31 03:54:40.130196539 +0000 UTC m=+1529.964056923"
Oct 31 03:54:40.184292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.184260    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:40.193590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.193560    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62h4f\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.193727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.193680    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.193727 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.193708    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.193818 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.193777    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.193917 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.193896    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.193928    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194020    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194059    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194089    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194338    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194399    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194775 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194444    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.194868 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.194846    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.195906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.195881    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.196796 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.196770    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.196902 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.196778    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.259584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.259542    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62h4f\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.261540 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.261509    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.350862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.348823    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:40.382051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.382019    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:40.384282 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.384236    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:40.384412 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.384319    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:40.589157 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:54:40.589122    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7064811_4b4b_4608_aa2a_5469c6d2571f.slice/crio-3ff71cfe5600b8dd172e4b29a3686cd70bedbd85238807c07d2535b4adbb1bbd WatchSource:0}: Error finding container 3ff71cfe5600b8dd172e4b29a3686cd70bedbd85238807c07d2535b4adbb1bbd: Status 404 returned error can't find the container with id 3ff71cfe5600b8dd172e4b29a3686cd70bedbd85238807c07d2535b4adbb1bbd
Oct 31 03:54:40.591760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.591735    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"]
Oct 31 03:54:40.592658 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.592620    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:40.592728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.592710    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:40.592779 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.592763    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:40.774110 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.774061    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"]
Oct 31 03:54:40.774833 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:54:40.774797    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1fb0e3b_e845_472d_8c2b_d6a718e8715e.slice/crio-d8f51f28daed6e221f370338be0dd5d78a2c83175fa6b910b9cecb9bebad7267 WatchSource:0}: Error finding container d8f51f28daed6e221f370338be0dd5d78a2c83175fa6b910b9cecb9bebad7267: Status 404 returned error can't find the container with id d8f51f28daed6e221f370338be0dd5d78a2c83175fa6b910b9cecb9bebad7267
Oct 31 03:54:40.778030 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.777990    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:40.778276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.778256    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:40.778389 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:40.778375    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:41.006484 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.006445    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" event={"ID":"a1fb0e3b-e845-472d-8c2b-d6a718e8715e","Type":"ContainerStarted","Data":"9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8"}
Oct 31 03:54:41.006905 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.006505    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" event={"ID":"a1fb0e3b-e845-472d-8c2b-d6a718e8715e","Type":"ContainerStarted","Data":"d8f51f28daed6e221f370338be0dd5d78a2c83175fa6b910b9cecb9bebad7267"}
Oct 31 03:54:41.008565 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.008535    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" event={"ID":"a7064811-4b4b-4608-aa2a-5469c6d2571f","Type":"ContainerStarted","Data":"03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3"}
Oct 31 03:54:41.008711 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.008592    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" event={"ID":"a7064811-4b4b-4608-aa2a-5469c6d2571f","Type":"ContainerStarted","Data":"3ff71cfe5600b8dd172e4b29a3686cd70bedbd85238807c07d2535b4adbb1bbd"}
Oct 31 03:54:41.139886 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.139192    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podStartSLOduration=2.139172372 podStartE2EDuration="2.139172372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:41.135180082 +0000 UTC m=+1530.969040412" watchObservedRunningTime="2026-10-31 03:54:41.139172372 +0000 UTC m=+1530.973032679"
Oct 31 03:54:41.184503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.184465    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:41.186485 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.186454    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:41.186673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.186518    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:41.289082 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.288959    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podStartSLOduration=2.288938593 podStartE2EDuration="2.288938593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:41.286347233 +0000 UTC m=+1531.120207551" watchObservedRunningTime="2026-10-31 03:54:41.288938593 +0000 UTC m=+1531.122798900"
Oct 31 03:54:41.372252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.368551    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:41.372252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.371327    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:41.372252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.371388    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:41.399053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.398997    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:41.399271 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:41.399177    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:42.192230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:42.191387    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:42.192230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:42.191448    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:42.366231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:42.364287    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:42.366231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:42.364348    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:42.391228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:42.390981    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:42.391228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:42.391052    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:43.192075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:43.192029    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:43.210359 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:43.210310    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:43.354228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:43.352608    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:43.354228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:43.352688    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:43.385231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:43.384784    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:43.385231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:43.384845    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:44.201280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.190257    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:44.201280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.190319    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:44.201280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.191191    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:44.370233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.351653    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:44.370233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.351717    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:44.401256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.383739    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:44.401256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:44.383798    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:45.204342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:45.204277    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:45.229516 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:45.204608    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:45.370351 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:45.370319    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:45.371707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:45.370871    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:45.392631 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:45.392509    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:45.392915 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:45.392883    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:46.091230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.090315    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:46.188237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.188066    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:46.188237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.188134    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:46.235114 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.235081    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:46.362397 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.361298    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:46.362397 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.361363    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:46.410965 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.410926    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:46.411383 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.410988    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:46.517518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:46.508617    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:47.075083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.074269    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"]
Oct 31 03:54:47.252233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.187576    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:47.252589 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.189442    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:47.252776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.252748    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:47.326842 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.326762    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"]
Oct 31 03:54:47.341083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.328410    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.482189 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.482146    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-af5cff2a-openshift-default-dockercfg-99dvb\""
Oct 31 03:54:47.512796 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.499115    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:47.512796 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.499171    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:47.512796 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.502679    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:47.515615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.515500    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:54:47.515615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.515553    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 31 03:54:47.817245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.814323    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:47.971086 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.970990    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.973114 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.973087    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.973320 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.973300    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.973733 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.973707    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.973904 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.973889    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.974051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.974038    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.974162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.974145    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.974404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.974385    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:47.974535 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:47.974521    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vgtl\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.076657 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.076620    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.076657 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.076667    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vgtl\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.076935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.076784    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.076935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.076860    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.076935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.076886    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.077091 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.076992    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.077091 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.077045    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.077193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.077100    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.077193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.077127    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.077760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.077734    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.078046 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.078020    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.078642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.078618    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.078995 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.078943    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.079536 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.079508    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.102987 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.102950    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.121415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.121298    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.142135 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.142102    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.142425 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.142395    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vgtl\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl\") pod \"e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") " pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.146402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.146154    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:48.195932 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.195900    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:54:48.196252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.196173    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 31 03:54:48.375690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.375023    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 31 03:54:48.375690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.375110    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 31 03:54:48.634756 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.634520    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:48.635934 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.635913    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:48.656643 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.656602    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 03:54:48.698750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.698721    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"]
Oct 31 03:54:48.699940 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:54:48.699916    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62bb728_1e8f_4a75_bca4_bdc12ddecb5f.slice/crio-fd9a9900fb5d3d5f910d139eb8ac66a193eec99f7f567c60e2d4a2f8a196c30b WatchSource:0}: Error finding container fd9a9900fb5d3d5f910d139eb8ac66a193eec99f7f567c60e2d4a2f8a196c30b: Status 404 returned error can't find the container with id fd9a9900fb5d3d5f910d139eb8ac66a193eec99f7f567c60e2d4a2f8a196c30b
Oct 31 03:54:48.703993 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.703286    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:48.703993 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.703370    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:48.703993 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:48.703413    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:49.277149 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.277111    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" event={"ID":"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f","Type":"ContainerStarted","Data":"39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"}
Oct 31 03:54:49.339341 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.339142    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" event={"ID":"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f","Type":"ContainerStarted","Data":"fd9a9900fb5d3d5f910d139eb8ac66a193eec99f7f567c60e2d4a2f8a196c30b"}
Oct 31 03:54:49.424342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.424285    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podStartSLOduration=3.424269252 podStartE2EDuration="3.424269252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:49.420490755 +0000 UTC m=+1539.254351059" watchObservedRunningTime="2026-10-31 03:54:49.424269252 +0000 UTC m=+1539.258129557"
Oct 31 03:54:49.550034 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.549944    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:49.552128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.552101    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:49.566579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.566544    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 03:54:49.684578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.684547    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:49.691474 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.686316    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:49.701232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:49.693459    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 03:54:50.146853 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:50.146810    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:50.152993 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:50.152964    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:50.182633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:50.182598    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:50.342230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:50.337427    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:50.394908 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:50.394880    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:54:51.011704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:51.011668    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:51.165786 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:51.165750    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:51.311706 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:51.311621    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:51.467930 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:51.467893    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:51.624771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:51.624684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:51.787938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:51.787905    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:52.150230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.148011    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"]
Oct 31 03:54:52.161671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.161610    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.230259 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.222793    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:52.254110 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.254070    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"]
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338388    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338450    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338520    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338573    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zlnx\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338701    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338770    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338808    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.338886    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.340725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.339030    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.364241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.358066    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-34a72a1f-gw-openshift-default-dockercfg-frwnb\""
Oct 31 03:54:52.420333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.389434    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:52.458092 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458051    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458125    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zlnx\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458274    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458353    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458402 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458391    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458468    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458604    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458645    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.458701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.458685    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.470219 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.467863    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.486606 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.474641    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.486820 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.475549    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.486926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.475608    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.487023 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.477016    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.487135 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.486524    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.538273 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.537850    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.562794 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.562740    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:52.570221 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.570143    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.574912 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.574614    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zlnx\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx\") pod \"e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") " pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.665326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.664916    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:52.913855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:52.913815    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:53.176591 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.176551    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"]
Oct 31 03:54:53.218228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.215627    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:53.218228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.215733    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:53.218228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.215776    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:53.482788 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.482749    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" event={"ID":"3e29db75-d021-42f2-b9d0-49bc35b99864","Type":"ContainerStarted","Data":"154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"}
Oct 31 03:54:53.482950 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.482804    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" event={"ID":"3e29db75-d021-42f2-b9d0-49bc35b99864","Type":"ContainerStarted","Data":"566447972dae76c0fc001222c3e239a72236d20504c32cc705d23edad8b555fb"}
Oct 31 03:54:53.608111 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.608053    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podStartSLOduration=1.6080373140000002 podStartE2EDuration="1.608037314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:53.607163766 +0000 UTC m=+1543.441024071" watchObservedRunningTime="2026-10-31 03:54:53.608037314 +0000 UTC m=+1543.441897618"
Oct 31 03:54:53.666012 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:53.665913    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:54.192307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.192272    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"]
Oct 31 03:54:54.196670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.196650    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.254373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.254197    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255141 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255102    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255167    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255253    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255278    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255435    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255466    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjcvs\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255596 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255491    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.255648 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.255588    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.278737 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.278697    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"]
Oct 31 03:54:54.286003 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.285966    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:54.303255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.303227    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-984fb1f2-openshift-default-dockercfg-94hlz\""
Oct 31 03:54:54.364416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.364360    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.364789 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.364760    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.365201 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.365179    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.365419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.365388    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.365603 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.365573    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.365817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.365795    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.365951 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.365932    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.366191 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.366160    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.366361 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.366338    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjcvs\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.366938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.365891    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.373322 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.367627    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.373579 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.367856    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.373730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.368058    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.395259 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.381397    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.405780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.396786    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.423135 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.423096    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.453475 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.452721    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjcvs\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.522562 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.487061    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token\") pod \"e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") " pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.522562 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.516654    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:54.529432 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.529391    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:54.674117 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:54.674077    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:55.025133 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:54:55.025095    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ee9a10_bd94_44ac_b4d8_2038181b18d5.slice/crio-5d35693ad422f6217f63817f8ba521648b94a2a835ebf1527df802bd8600ad68 WatchSource:0}: Error finding container 5d35693ad422f6217f63817f8ba521648b94a2a835ebf1527df802bd8600ad68: Status 404 returned error can't find the container with id 5d35693ad422f6217f63817f8ba521648b94a2a835ebf1527df802bd8600ad68
Oct 31 03:54:55.026260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.026232    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"]
Oct 31 03:54:55.027555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.027519    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:55.027639 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.027603    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:55.027639 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.027632    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:54:55.493717 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.493676    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:55.634230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.630074    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" event={"ID":"f4ee9a10-bd94-44ac-b4d8-2038181b18d5","Type":"ContainerStarted","Data":"cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"}
Oct 31 03:54:55.634230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.630141    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" event={"ID":"f4ee9a10-bd94-44ac-b4d8-2038181b18d5","Type":"ContainerStarted","Data":"5d35693ad422f6217f63817f8ba521648b94a2a835ebf1527df802bd8600ad68"}
Oct 31 03:54:55.634230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.630178    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:55.661456 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.661358    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:54:55.757589 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:55.757465    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podStartSLOduration=1.7574495049999999 podStartE2EDuration="1.757449505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:55.754782824 +0000 UTC m=+1545.588643154" watchObservedRunningTime="2026-10-31 03:54:55.757449505 +0000 UTC m=+1545.591309809"
Oct 31 03:54:56.135326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:56.135237    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:56.296703 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:56.296672    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:56.520304 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:56.520270    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:56.525572 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:56.525546    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:56.635271 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:56.635240    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:56.636261 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:56.636241    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:54:58.866237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:58.865396    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:58.908240 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:58.887888    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.084232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.083395    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:59.270707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.248778    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:59.455227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.446189    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.519283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.518372    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.519467 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.519434    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"]
Oct 31 03:54:59.548355 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.537448    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:59.727232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.719472    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.878914 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.878811    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:54:59.903695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.903467    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.906889 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.906857    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:54:59.912231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:54:59.907542    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:00.070444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.040079    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-34a72a1f\""
Oct 31 03:55:00.070444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.043532    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-34a72a1f-dockercfg-2f5x8\""
Oct 31 03:55:00.204730 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.204633    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:00.208952 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.208742    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.231346 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.231113    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.452229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.449056    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:00.511349 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.511288    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.572403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.539931    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4j6q\" (UniqueName: \"kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q\") pod \"payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") " pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:00.590460 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.587028    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume\") pod \"payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") " pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:00.776285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.697748    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume\") pod \"payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") " pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:00.776285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.698108    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4j6q\" (UniqueName: \"kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q\") pod \"payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") " pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:00.806234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:00.805630    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume\") pod \"payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") " pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:01.105290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.089147    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4j6q\" (UniqueName: \"kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q\") pod \"payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") " pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:01.152328 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.116402    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:55:01.180862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.162044    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:55:01.430653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.430558    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"]
Oct 31 03:55:01.616358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.206898    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:01.616358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.182555    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:01.616358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.545805    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": context deadline exceeded"
Oct 31 03:55:01.778361 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.778313    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.880245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.814679    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 31 03:55:01.890769 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.816280    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 31 03:55:01.907281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:01.870649    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:02.126632 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.122343    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:02.126632 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.122417    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.375081 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.374901    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:02.443255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.431586    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:02.443255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.431654    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.879371 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.872363    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:02.879371 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.872434    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.909233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.883626    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.909233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.887798    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.909233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.894499    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.909233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.894916    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:55:02.939235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:02.930730    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:55:03.684291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:03.683355    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 03:55:03.908197 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:03.899337    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.936757 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:03.936648    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 31 03:55:04.957254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:04.952720    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:04.957254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:04.953266    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 03:55:04.981278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:04.972000    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:05.220659 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:05.198125    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"]
Oct 31 03:55:05.272266 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:05.244380    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 03:55:05.368363 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:05.363483    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 03:55:05.570264 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:05.568750    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76" event={"ID":"ab34ecaf-1e52-4fc0-97a9-501eece79c03","Type":"ContainerStarted","Data":"1df0540410904662a19d68254cc6cb1ef6beb86dfda109d67ec9ba7a1951814e"}
Oct 31 03:55:06.008527 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:06.008492    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:06.381800 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:06.381719    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:55:06.577368 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:06.577332    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76" event={"ID":"ab34ecaf-1e52-4fc0-97a9-501eece79c03","Type":"ContainerStarted","Data":"1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8"}
Oct 31 03:55:06.577580 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:06.577395    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:06.698523 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:06.698433    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76" podStartSLOduration=7.698418479 podStartE2EDuration="7.698418479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:06.696309156 +0000 UTC m=+1556.530169464" watchObservedRunningTime="2026-10-31 03:55:06.698418479 +0000 UTC m=+1556.532278784"
Oct 31 03:55:10.624720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:10.624685    5906 scope.go:122] "RemoveContainer" containerID="5178a1312375f27e6ebefa26312bbe48aff05adc41b91663ba86baf708b5b0a7"
Oct 31 03:55:11.113938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:11.113900    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:13.358644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:13.357872    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:13.521158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:13.521123    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:13.786778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:13.786727    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"]
Oct 31 03:55:14.000572 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.000529    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"]
Oct 31 03:55:14.000943 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.000919    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.050283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.033528    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"]
Oct 31 03:55:14.064457 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.064429    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.082404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.082266    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"]
Oct 31 03:55:14.163568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.161329    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.163568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.161484    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzntz\" (UniqueName: \"kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz\") pod \"payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.163568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.161848    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-984fb1f2\""
Oct 31 03:55:14.163568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.161980    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-984fb1f2-dockercfg-2nhtc\""
Oct 31 03:55:14.263193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.263154    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume\") pod \"payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") " pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.263424 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.263335    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72mnw\" (UniqueName: \"kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw\") pod \"payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") " pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.263508 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.263489    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.263651 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.263630    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzntz\" (UniqueName: \"kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz\") pod \"payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.264624 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.264591    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.328084 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.328013    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzntz\" (UniqueName: \"kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz\") pod \"payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.364881 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.364847    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72mnw\" (UniqueName: \"kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw\") pod \"payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") " pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.365295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.365040    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume\") pod \"payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") " pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.365739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.365716    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume\") pod \"payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") " pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.407296 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.407264    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:14.427655 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.427628    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72mnw\" (UniqueName: \"kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw\") pod \"payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") " pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.465810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.465784    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:14.565300 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.565247    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:14.714881 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.714798    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:14.780087 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:14.780059    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5e93eb_f9d0_45a9_8924_35abafd0ce9e.slice/crio-8530c2c000a1f4f001af76efa14467c501f08dbea103869fbac4613220af2270 WatchSource:0}: Error finding container 8530c2c000a1f4f001af76efa14467c501f08dbea103869fbac4613220af2270: Status 404 returned error can't find the container with id 8530c2c000a1f4f001af76efa14467c501f08dbea103869fbac4613220af2270
Oct 31 03:55:14.783885 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.783864    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"]
Oct 31 03:55:14.854636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.854606    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:14.885897 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:14.885861    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490f9cd4_26c0_48b2_8ff5_9a8fecc5ec8f.slice/crio-9ac90bff2f4ad1ffa722693b0bcad523735e0872276785c4dea989410fc20637 WatchSource:0}: Error finding container 9ac90bff2f4ad1ffa722693b0bcad523735e0872276785c4dea989410fc20637: Status 404 returned error can't find the container with id 9ac90bff2f4ad1ffa722693b0bcad523735e0872276785c4dea989410fc20637
Oct 31 03:55:14.888999 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:14.888972    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"]
Oct 31 03:55:15.636763 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.636731    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:15.728192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.728151    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv" event={"ID":"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f","Type":"ContainerStarted","Data":"ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780"}
Oct 31 03:55:15.728192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.728227    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:15.728497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.728261    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv" event={"ID":"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f","Type":"ContainerStarted","Data":"9ac90bff2f4ad1ffa722693b0bcad523735e0872276785c4dea989410fc20637"}
Oct 31 03:55:15.729962 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.729930    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h" event={"ID":"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e","Type":"ContainerStarted","Data":"994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4"}
Oct 31 03:55:15.730107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.729979    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h" event={"ID":"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e","Type":"ContainerStarted","Data":"8530c2c000a1f4f001af76efa14467c501f08dbea103869fbac4613220af2270"}
Oct 31 03:55:15.730107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.730027    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:15.790115 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.790086    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:15.950283 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:15.907935    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv" podStartSLOduration=2.907916294 podStartE2EDuration="2.907916294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:15.907149156 +0000 UTC m=+1565.741009462" watchObservedRunningTime="2026-10-31 03:55:15.907916294 +0000 UTC m=+1565.741776600"
Oct 31 03:55:16.070005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.061197    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:16.151232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.139512    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h" podStartSLOduration=3.139488494 podStartE2EDuration="3.139488494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:16.059692606 +0000 UTC m=+1565.893552912" watchObservedRunningTime="2026-10-31 03:55:16.139488494 +0000 UTC m=+1565.973348800"
Oct 31 03:55:16.151232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.141434    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"]
Oct 31 03:55:16.151232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.151198    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.180051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.179360    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46vlr\" (UniqueName: \"kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr\") pod \"payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.180051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.179527    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.202391 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.202155    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:16.209588 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.209542    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-af5cff2a\""
Oct 31 03:55:16.213976 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.213928    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-af5cff2a-dockercfg-xrx9t\""
Oct 31 03:55:16.214771 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.214743    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"]
Oct 31 03:55:16.414246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.398800    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:16.414246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.405355    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46vlr\" (UniqueName: \"kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr\") pod \"payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.414246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.405518    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.414246 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.407544    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.590642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.590380    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46vlr\" (UniqueName: \"kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr\") pod \"payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.654275 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.653498    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"]
Oct 31 03:55:16.671530 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.665370    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.674780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.674747    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"]
Oct 31 03:55:16.707165 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.707124    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume\") pod \"payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") " pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.707373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.707343    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4lw6\" (UniqueName: \"kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6\") pod \"payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") " pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.809264 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.809183    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4lw6\" (UniqueName: \"kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6\") pod \"payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") " pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.809474 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.809350    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:16.809549 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.809500    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume\") pod \"payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") " pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.810175 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.810156    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume\") pod \"payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") " pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.873504 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.873462    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4lw6\" (UniqueName: \"kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6\") pod \"payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") " pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:16.986928 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:16.986893    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:17.185572 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:17.185540    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b629681_070f_4eb6_93a4_95467c37643e.slice/crio-d8b1d3c640611a3a0d523f4c41425af2b3d0f4b503c7ef2c363c4358e5ba9a22 WatchSource:0}: Error finding container d8b1d3c640611a3a0d523f4c41425af2b3d0f4b503c7ef2c363c4358e5ba9a22: Status 404 returned error can't find the container with id d8b1d3c640611a3a0d523f4c41425af2b3d0f4b503c7ef2c363c4358e5ba9a22
Oct 31 03:55:17.190223 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.190189    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"]
Oct 31 03:55:17.363560 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:17.363523    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab59f81_9832_4aa6_9956_3320c7dbed72.slice/crio-8b34b68372d326d2480bcf6ae9b902c26e541e17ca667bc1b1c9ad64af14ada1 WatchSource:0}: Error finding container 8b34b68372d326d2480bcf6ae9b902c26e541e17ca667bc1b1c9ad64af14ada1: Status 404 returned error can't find the container with id 8b34b68372d326d2480bcf6ae9b902c26e541e17ca667bc1b1c9ad64af14ada1
Oct 31 03:55:17.366938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.366907    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"]
Oct 31 03:55:17.587842 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.587809    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:55:17.745940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.745905    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn" event={"ID":"bab59f81-9832-4aa6-9956-3320c7dbed72","Type":"ContainerStarted","Data":"623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94"}
Oct 31 03:55:17.746392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.745956    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn" event={"ID":"bab59f81-9832-4aa6-9956-3320c7dbed72","Type":"ContainerStarted","Data":"8b34b68372d326d2480bcf6ae9b902c26e541e17ca667bc1b1c9ad64af14ada1"}
Oct 31 03:55:17.746392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.746127    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:17.747869 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.747843    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr" event={"ID":"3b629681-070f-4eb6-93a4-95467c37643e","Type":"ContainerStarted","Data":"87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6"}
Oct 31 03:55:17.747996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.747890    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr" event={"ID":"3b629681-070f-4eb6-93a4-95467c37643e","Type":"ContainerStarted","Data":"d8b1d3c640611a3a0d523f4c41425af2b3d0f4b503c7ef2c363c4358e5ba9a22"}
Oct 31 03:55:17.747996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.747916    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:17.869824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.869771    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn" podStartSLOduration=1.8697546040000002 podStartE2EDuration="1.869754604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:17.865947007 +0000 UTC m=+1567.699807313" watchObservedRunningTime="2026-10-31 03:55:17.869754604 +0000 UTC m=+1567.703614969"
Oct 31 03:55:17.987671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:17.987183    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr" podStartSLOduration=1.98715221 podStartE2EDuration="1.98715221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:17.986137178 +0000 UTC m=+1567.819997485" watchObservedRunningTime="2026-10-31 03:55:17.98715221 +0000 UTC m=+1567.821012516"
Oct 31 03:55:18.804378 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:18.804326    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:19.661600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:19.661564    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:19.800830 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:19.800796    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:20.094759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:20.094718    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:20.320184 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:20.320144    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:22.692964 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:22.692922    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:24.377488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.377451    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"]
Oct 31 03:55:24.383184 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.383157    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.429163 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.429119    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"]
Oct 31 03:55:24.433873 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.433845    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.481249 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.481186    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"]
Oct 31 03:55:24.531363 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.531328    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"]
Oct 31 03:55:24.563280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563244    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.563280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563280    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.563531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563451    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.563531 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563518    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.563633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563536    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.563633 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563592    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.563743 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563631    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.563800 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563758    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.563855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563794    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.563971 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.563951    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.564020 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564003    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.564085 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564072    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.564131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564118    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qtk8\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.564178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564146    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.564260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564200    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.564310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564274    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.564310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564297    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74wfz\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.564382 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.564341    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.665887 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.665778    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.665887 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.665853    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.665909    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qtk8\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666018    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666072    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666131 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666125    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74wfz\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666182    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666338    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666422    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666512    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666546    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666575    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666775 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666664    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666775 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666693    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666775 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666700    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.666923 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666772    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.666923 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666835    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.667045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.666970    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.667045 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667006    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.667150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667113    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.667150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667125    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.667150 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667138    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.667312 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667300    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.667493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667473    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.667575 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667554    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.667720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667694    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.667772 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.667701    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.668764 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.668734    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.668912 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.668891    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.669350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.669330    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.669716 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.669694    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.741263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.741228    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.741453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.741399    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74wfz\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:24.742764 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.742743    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qtk8\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.743376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.743064    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.748125 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.748104    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:24.901807 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.847832    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:24.999350 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:24.999320    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:25.149660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.149618    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:25.178595 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:25.178559    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode41e8673_22e9_4fd9_b3b2_43154730fc18.slice/crio-da35decce63ff9028fcaab37509697b49770c4e2a8a601a700281957e3b65b32 WatchSource:0}: Error finding container da35decce63ff9028fcaab37509697b49770c4e2a8a601a700281957e3b65b32: Status 404 returned error can't find the container with id da35decce63ff9028fcaab37509697b49770c4e2a8a601a700281957e3b65b32
Oct 31 03:55:25.181400 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.181373    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"]
Oct 31 03:55:25.181490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.181423    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:25.181490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.181486    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:25.181570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.181514    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:25.397816 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:25.397768    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96ac7cb_59ab_4824_9c9f_6439c5d9b113.slice/crio-01cbba417705d36a114f0fe917fe47135517d9b298d184caa149b884e0368755 WatchSource:0}: Error finding container 01cbba417705d36a114f0fe917fe47135517d9b298d184caa149b884e0368755: Status 404 returned error can't find the container with id 01cbba417705d36a114f0fe917fe47135517d9b298d184caa149b884e0368755
Oct 31 03:55:25.402581 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.402525    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:25.402712 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.402685    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:25.402800 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.402776    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:25.406349 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.406319    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"]
Oct 31 03:55:25.814374 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.814332    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" event={"ID":"e41e8673-22e9-4fd9-b3b2-43154730fc18","Type":"ContainerStarted","Data":"232c82461e381aa65418c6dc7d7dbf94d235e0bdb8de1d99aff5bc6281ed8b79"}
Oct 31 03:55:25.814566 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.814398    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" event={"ID":"e41e8673-22e9-4fd9-b3b2-43154730fc18","Type":"ContainerStarted","Data":"da35decce63ff9028fcaab37509697b49770c4e2a8a601a700281957e3b65b32"}
Oct 31 03:55:25.816392 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.816360    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" event={"ID":"b96ac7cb-59ab-4824-9c9f-6439c5d9b113","Type":"ContainerStarted","Data":"7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c"}
Oct 31 03:55:25.816545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.816414    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" event={"ID":"b96ac7cb-59ab-4824-9c9f-6439c5d9b113","Type":"ContainerStarted","Data":"01cbba417705d36a114f0fe917fe47135517d9b298d184caa149b884e0368755"}
Oct 31 03:55:25.941955 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.941901    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podStartSLOduration=1.941886744 podStartE2EDuration="1.941886744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:25.938450199 +0000 UTC m=+1575.772310503" watchObservedRunningTime="2026-10-31 03:55:25.941886744 +0000 UTC m=+1575.775747083"
Oct 31 03:55:25.999967 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:25.999917    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:26.071172 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.071057    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podStartSLOduration=2.071042456 podStartE2EDuration="2.071042456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:26.069184531 +0000 UTC m=+1575.903044873" watchObservedRunningTime="2026-10-31 03:55:26.071042456 +0000 UTC m=+1575.904902755"
Oct 31 03:55:26.464311 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.464156    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:26.614634 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.614602    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:26.739528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.739451    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:55:26.739528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.739522    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:55:26.763228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.749624    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:26.763228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.757356    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:26.763228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.757420    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:26.763228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:26.762115    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:27.024226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:27.024165    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:27.024438 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:27.024260    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:27.077226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:27.076917    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:27.750230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:27.749635    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:27.750230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:27.749713    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:28.003288 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:28.001585    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:28.003288 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:28.001650    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:28.792231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:28.789533    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:28.792231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:28.789597    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:28.800220 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:28.800109    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:55:28.806144 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:28.806121    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:55:29.001230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:29.000268    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:29.001230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:29.000346    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:29.394381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:29.394316    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:29.753231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:29.749661    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:29.753231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:29.749729    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:30.004321 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:30.003889    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:30.004321 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:30.003953    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:30.760231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:30.749641    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:30.760231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:30.749705    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:31.000056 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:31.000020    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:31.000261 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:31.000080    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:31.776233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:31.761962    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:31.776233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:31.762028    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:32.000644 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:32.000604    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:32.000834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:32.000671    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:32.749930 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:32.749483    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 31 03:55:32.749930 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:32.749542    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 31 03:55:33.001473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:33.001430    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 31 03:55:33.002115 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:33.002074    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 31 03:55:33.680972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:33.680908    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:33.856231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:33.838092    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:34.443050 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:34.442983    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:34.833456 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:34.833405    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:34.833803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:34.833751    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:35.089453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.089348    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:35.089716 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.089685    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:35.371468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.371355    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:35.371762 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.371709    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:35.845231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.825728    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:35.845231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.827480    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:35.845231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.834549    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:35.845231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.836008    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:35.860237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.849515    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 03:55:35.860237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:35.849905    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 03:55:36.511511 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:36.511468    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:55:38.569457 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.569420    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"]
Oct 31 03:55:38.574331 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.574303    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.629950 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.629915    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"]
Oct 31 03:55:38.677374 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.677341    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-34a72a1f-serving-cert\""
Oct 31 03:55:38.690263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.690233    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glc9d\" (UniqueName: \"kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.690451 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.690410    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.690509 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.690474    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.690560 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.690506    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.752636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.752600    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:38.792077 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.792037    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.792275 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.792123    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.792275 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.792156    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.792275 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.792243    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glc9d\" (UniqueName: \"kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.792387 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:38.792282    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-serving-cert: secret "maas-api-e2e-migrate-34a72a1f-serving-cert" not found
Oct 31 03:55:38.792387 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:38.792372    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls podName:3b54e29e-b8c6-458b-a983-afa7824856bf nodeName:}" failed. No retries permitted until 2026-10-31 03:55:39.292348134 +0000 UTC m=+1589.126208431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls") pod "maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" (UID: "3b54e29e-b8c6-458b-a983-afa7824856bf") : secret "maas-api-e2e-migrate-34a72a1f-serving-cert" not found
Oct 31 03:55:38.792773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.792750    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.794725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.794702    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.875471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.875367    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glc9d\" (UniqueName: \"kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:38.920226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:38.917842    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:39.143845 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.143736    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:39.311016 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.310979    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:39.313491 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.313470    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\") pod \"maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:39.323750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.323716    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:39.489552 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.489468    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:39.868765 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:39.868727    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b54e29e_b8c6_458b_a983_afa7824856bf.slice/crio-e21f5f7ca920853dbc5a3fcfa3a8da1013dfb89ce7efcf422547b7a34e0e3f99 WatchSource:0}: Error finding container e21f5f7ca920853dbc5a3fcfa3a8da1013dfb89ce7efcf422547b7a34e0e3f99: Status 404 returned error can't find the container with id e21f5f7ca920853dbc5a3fcfa3a8da1013dfb89ce7efcf422547b7a34e0e3f99
Oct 31 03:55:39.871286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.871266    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:55:39.872336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:39.872314    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"]
Oct 31 03:55:40.893287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:40.893201    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" event={"ID":"3b54e29e-b8c6-458b-a983-afa7824856bf","Type":"ContainerStarted","Data":"b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da"}
Oct 31 03:55:40.893761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:40.893345    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" event={"ID":"3b54e29e-b8c6-458b-a983-afa7824856bf","Type":"ContainerStarted","Data":"e21f5f7ca920853dbc5a3fcfa3a8da1013dfb89ce7efcf422547b7a34e0e3f99"}
Oct 31 03:55:40.893761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:40.893467    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:41.030099 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:41.030033    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" podStartSLOduration=2.179467642 podStartE2EDuration="3.030013661s" totalImagesPullingTime="850.546019ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:41.02728504 +0000 UTC m=+1590.861145344" watchObservedRunningTime="2026-10-31 03:55:41.030013661 +0000 UTC m=+1590.863873968"
Oct 31 03:55:43.197343 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:43.197308    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:45.375514 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:45.375483    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:46.907076 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:46.907047    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:55:47.561536 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.561491    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"]
Oct 31 03:55:47.567483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.567456    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.621813 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.621782    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-984fb1f2-serving-cert\""
Oct 31 03:55:47.622979 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.622942    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"]
Oct 31 03:55:47.669659 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.669617    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.669867 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.669676    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.669963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.669940    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.670099 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.670083    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hhst\" (UniqueName: \"kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.737172 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.737134    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:47.771332 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.771292    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hhst\" (UniqueName: \"kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.771528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.771359    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.771528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.771400    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.771528 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:47.771512    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-serving-cert: secret "maas-api-e2e-ait-984fb1f2-serving-cert" not found
Oct 31 03:55:47.771665 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.771575    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.771665 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:47.771587    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls podName:07f23535-407b-46c4-90cd-01979f4302b1 nodeName:}" failed. No retries permitted until 2026-10-31 03:55:48.271570456 +0000 UTC m=+1598.105430760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls") pod "maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" (UID: "07f23535-407b-46c4-90cd-01979f4302b1") : secret "maas-api-e2e-ait-984fb1f2-serving-cert" not found
Oct 31 03:55:47.772266 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.772195    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.774091 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.774072    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.849767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.849670    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hhst\" (UniqueName: \"kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:47.930232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:47.922254    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:48.072683 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:48.072607    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:48.368333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:48.368294    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:48.370769 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:48.370747    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\") pod \"maas-api-e2e-ait-984fb1f2-8558659475-vl2zh\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:48.482961 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:48.482924    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:48.860250 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:48.860195    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07f23535_407b_46c4_90cd_01979f4302b1.slice/crio-364886a25000338563c4612ef693ac810b28d39ecc5ef029b10698019b5c52f7 WatchSource:0}: Error finding container 364886a25000338563c4612ef693ac810b28d39ecc5ef029b10698019b5c52f7: Status 404 returned error can't find the container with id 364886a25000338563c4612ef693ac810b28d39ecc5ef029b10698019b5c52f7
Oct 31 03:55:48.863506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:48.863478    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"]
Oct 31 03:55:48.991619 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:48.991584    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" event={"ID":"07f23535-407b-46c4-90cd-01979f4302b1","Type":"ContainerStarted","Data":"364886a25000338563c4612ef693ac810b28d39ecc5ef029b10698019b5c52f7"}
Oct 31 03:55:50.000879 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:50.000841    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" event={"ID":"07f23535-407b-46c4-90cd-01979f4302b1","Type":"ContainerStarted","Data":"6ae330bf4d953bdfb6011c043d1ec16ead55489f9df8a283699d9b632d59448a"}
Oct 31 03:55:50.001309 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:50.001050    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:50.125366 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:50.125296    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" podStartSLOduration=2.286649005 podStartE2EDuration="3.125275687s" totalImagesPullingTime="838.626682ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:50.120972803 +0000 UTC m=+1599.954833108" watchObservedRunningTime="2026-10-31 03:55:50.125275687 +0000 UTC m=+1599.959135993"
Oct 31 03:55:52.103827 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.103790    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"]
Oct 31 03:55:52.108523 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.108502    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.168864 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.168827    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"]
Oct 31 03:55:52.211316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.211277    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-af5cff2a-serving-cert\""
Oct 31 03:55:52.250446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.250413    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwl72\" (UniqueName: \"kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.250617 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.250487    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.250719 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.250689    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.250791 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.250775    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.283926 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.283897    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:52.352619 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.352588    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwl72\" (UniqueName: \"kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.352821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.352643    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.352821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.352715    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.352821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.352745    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.353025 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:52.353005    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-serving-cert: secret "maas-api-e2e-ait-af5cff2a-serving-cert" not found
Oct 31 03:55:52.353092 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:55:52.353080    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls podName:be93559a-bd9d-47c2-a3e4-55377c6deaaa nodeName:}" failed. No retries permitted until 2026-10-31 03:55:52.853057174 +0000 UTC m=+1602.686917461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls") pod "maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" (UID: "be93559a-bd9d-47c2-a3e4-55377c6deaaa") : secret "maas-api-e2e-ait-af5cff2a-serving-cert" not found
Oct 31 03:55:52.353404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.353379    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.355307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.355257    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.415778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.415747    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwl72\" (UniqueName: \"kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.433459 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.433430    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:52.659573 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.659492    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:52.864671 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.864633    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:52.867161 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:52.867139    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\") pod \"maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:53.023163 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:53.023127    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:53.403057 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:53.403015    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe93559a_bd9d_47c2_a3e4_55377c6deaaa.slice/crio-59f5bcc814be2d45286f6440fc4df1337ac1a92d6afd42a583b80c07e283bf5c WatchSource:0}: Error finding container 59f5bcc814be2d45286f6440fc4df1337ac1a92d6afd42a583b80c07e283bf5c: Status 404 returned error can't find the container with id 59f5bcc814be2d45286f6440fc4df1337ac1a92d6afd42a583b80c07e283bf5c
Oct 31 03:55:53.405441 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:53.405412    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"]
Oct 31 03:55:54.029935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.029903    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" event={"ID":"be93559a-bd9d-47c2-a3e4-55377c6deaaa","Type":"ContainerStarted","Data":"59f5bcc814be2d45286f6440fc4df1337ac1a92d6afd42a583b80c07e283bf5c"}
Oct 31 03:55:54.324165 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.324138    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"]
Oct 31 03:55:54.330184 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.330047    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438325    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438407    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438506    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438664    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438709    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438819    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.438858 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.438850    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrgwr\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.439240 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.439112    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.439240 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.439139    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.456989 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.456948    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"]
Oct 31 03:55:54.541018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.540982    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541017    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541053    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541080    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541113    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541245    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541286    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541369    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541394    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrgwr\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541417    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541504    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541557    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.541843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.541736    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.542026 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.542001    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.543615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.543593    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.543767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.543748    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.604457 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.604386    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.604602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.604454    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrgwr\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr\") pod \"maas-default-gateway-openshift-default-7854949c9-mq99p\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:54.642037 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.642002    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:54.678720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:54.678685    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:55.063313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.061871    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" event={"ID":"be93559a-bd9d-47c2-a3e4-55377c6deaaa","Type":"ContainerStarted","Data":"dd777586030b7388737fe4b83b4a0a9baac4e74821d5db85507623df8495ceaf"}
Oct 31 03:55:55.063313 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.063105    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:55:55.176241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.169302    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"]
Oct 31 03:55:55.202284 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:55:55.202231    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2279d7a6_0e60_46c6_96b2_2c702eeb203b.slice/crio-ccf797723362cd07cfba441e6afc1cefe66b44bc48c1e7f90f3e6dcf159251cc WatchSource:0}: Error finding container ccf797723362cd07cfba441e6afc1cefe66b44bc48c1e7f90f3e6dcf159251cc: Status 404 returned error can't find the container with id ccf797723362cd07cfba441e6afc1cefe66b44bc48c1e7f90f3e6dcf159251cc
Oct 31 03:55:55.216555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.206044    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:55.216555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.206141    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:55.216555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.206187    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:55:55.216555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:55.207500    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" podStartSLOduration=2.311761634 podStartE2EDuration="3.207480161s" totalImagesPullingTime="895.718527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:55.195715137 +0000 UTC m=+1605.029575451" watchObservedRunningTime="2026-10-31 03:55:55.207480161 +0000 UTC m=+1605.041340467"
Oct 31 03:55:56.038813 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.038773    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:55:56.072674 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.072635    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" event={"ID":"2279d7a6-0e60-46c6-96b2-2c702eeb203b","Type":"ContainerStarted","Data":"aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044"}
Oct 31 03:55:56.072848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.072693    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" event={"ID":"2279d7a6-0e60-46c6-96b2-2c702eeb203b","Type":"ContainerStarted","Data":"ccf797723362cd07cfba441e6afc1cefe66b44bc48c1e7f90f3e6dcf159251cc"}
Oct 31 03:55:56.330773 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.330667    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podStartSLOduration=2.330649955 podStartE2EDuration="2.330649955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:56.328395274 +0000 UTC m=+1606.162255575" watchObservedRunningTime="2026-10-31 03:55:56.330649955 +0000 UTC m=+1606.164510259"
Oct 31 03:55:56.679761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.679678    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:55:56.681269 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.681242    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:55:56.681327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:56.681297    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:55:57.019861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:57.019821    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:55:57.679413 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:57.679374    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:55:57.679846 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:57.679435    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:55:58.680101 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:58.680062    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:55:58.680525 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:58.680121    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:55:59.679598 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:59.679561    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:55:59.679861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:55:59.679618    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:56:00.679823 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:00.679784    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:56:00.680300 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:00.679841    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:56:01.592540 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:01.592500    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:01.692089 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:01.692052    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:56:01.692745 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:01.692711    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:56:02.108231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:02.107646    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:56:02.683238 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:02.681499    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:56:02.683238 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:02.681590    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:56:03.681063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:03.680589    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:56:03.681063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:03.680655    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:56:04.680743 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:04.680700    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mq99p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 31 03:56:04.681048 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:04.681015    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 31 03:56:06.262409 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.262319    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:06.674894 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.674791    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:56:06.677371 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.677194    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:56:06.710841 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.710808    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 03:56:06.746289 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:56:06.746123    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:56:06.890679 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.889328    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:56:06.916417 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.916368    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:56:06.954458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:06.954324    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:56:10.674759 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:10.674719    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:11.670403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:11.670364    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:56:11.670610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:11.670431    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:56:12.677437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:12.677399    5906 patch_prober.go:29] interesting pod/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:56:12.677835 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:12.677463    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:56:13.552457 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.552423    5906 generic.go:356] "Generic (PLEG): container finished" podID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerID="cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9" exitCode=0
Oct 31 03:56:13.552679 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.552498    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" event={"ID":"f4ee9a10-bd94-44ac-b4d8-2038181b18d5","Type":"ContainerDied","Data":"cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"}
Oct 31 03:56:13.553090 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.553068    5906 scope.go:122] "RemoveContainer" containerID="cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"
Oct 31 03:56:13.554462 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.554439    5906 generic.go:356] "Generic (PLEG): container finished" podID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerID="154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82" exitCode=0
Oct 31 03:56:13.554583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.554509    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" event={"ID":"3e29db75-d021-42f2-b9d0-49bc35b99864","Type":"ContainerDied","Data":"154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"}
Oct 31 03:56:13.554889 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.554876    5906 scope.go:122] "RemoveContainer" containerID="154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"
Oct 31 03:56:13.554948 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.554923    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:13.555007 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.554995    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:13.555050 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.555030    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:13.556459 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.556431    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:13.556552 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.556497    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:13.556552 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:13.556539    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:14.599482 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:14.599445    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" event={"ID":"f4ee9a10-bd94-44ac-b4d8-2038181b18d5","Type":"ContainerStarted","Data":"4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3"}
Oct 31 03:56:14.602558 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:14.602529    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" event={"ID":"3e29db75-d021-42f2-b9d0-49bc35b99864","Type":"ContainerStarted","Data":"3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41"}
Oct 31 03:56:14.666227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:14.665679    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:56:14.666227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:14.665821    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 03:56:14.666227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:14.665864    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 03:56:15.192241 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:15.191961    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:15.520642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:15.520594    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:56:15.520922 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:15.520896    5906 patch_prober.go:29] interesting pod/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 03:56:15.520990 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:15.520947    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 03:56:15.666231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:15.665804    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 03:56:15.666231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:15.665867    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 03:56:16.520463 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:16.520426    5906 patch_prober.go:29] interesting pod/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 03:56:16.520710 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:16.520680    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 03:56:16.666231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:16.665737    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 03:56:16.666231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:16.665822    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 03:56:17.520780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:17.520744    5906 patch_prober.go:29] interesting pod/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 03:56:17.521003 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:17.520811    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 03:56:17.666699 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:17.666656    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 03:56:17.667321 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:17.667288    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 03:56:18.520294 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:18.520261    5906 patch_prober.go:29] interesting pod/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 03:56:18.520480 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:18.520311    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 03:56:18.669468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:18.669363    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 03:56:18.670091 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:18.670033    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 03:56:19.520381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:19.520342    5906 patch_prober.go:29] interesting pod/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 31 03:56:19.520678 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:19.520643    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 31 03:56:19.672239 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:19.668684    5906 patch_prober.go:29] interesting pod/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 31 03:56:19.672239 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:19.668752    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 31 03:56:19.796862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:19.796768    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:20.525412 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:20.525380    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:56:20.525679 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:20.525658    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:56:20.526511 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:20.526492    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:56:20.670395 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:20.670363    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:56:20.670681 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:20.670657    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:56:20.671590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:20.671569    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:56:23.038420 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:23.038387    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:24.305625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:24.305592    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:28.818739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:28.818701    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:33.087439 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:33.087404    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:36.383622 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:56:36.383581    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:56:37.994284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:37.994248    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:42.838174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:42.838139    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:46.860945 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:46.860868    5906 generic.go:356] "Generic (PLEG): container finished" podID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerID="39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c" exitCode=0
Oct 31 03:56:46.861405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:46.860943    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" event={"ID":"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f","Type":"ContainerDied","Data":"39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"}
Oct 31 03:56:46.861478 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:46.861460    5906 scope.go:122] "RemoveContainer" containerID="39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"
Oct 31 03:56:46.863267 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:46.863236    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:46.863384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:46.863315    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:46.863384 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:46.863349    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:56:47.870625 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:47.870586    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" event={"ID":"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f","Type":"ContainerStarted","Data":"2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3"}
Oct 31 03:56:47.987410 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:47.987380    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:48.151279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:48.147815    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:56:48.151279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:48.149084    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:48.151279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:48.149142    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:48.966232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:48.963776    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:49.108939 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:49.108907    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:49.158469 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:49.158431    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:49.158766 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:49.158730    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:49.219299 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:49.219191    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"]
Oct 31 03:56:49.567163 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:49.567118    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:49.709488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:49.709444    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:50.160228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:50.156686    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:50.160228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:50.156766    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:50.337681 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:50.337649    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:56:51.163231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:51.160950    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:51.163231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:51.161016    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:52.160231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:52.153667    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:52.160231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:52.153748    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:52.821927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:52.821840    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:53.166294 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:53.154801    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:53.166294 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:53.154876    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:54.164232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.154693    5906 patch_prober.go:29] interesting pod/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 31 03:56:54.164232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.154779    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 31 03:56:54.219566 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.219519    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76" podUID="ab34ecaf-1e52-4fc0-97a9-501eece79c03" containerName="payload-processing" containerID="cri-o://1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8" gracePeriod=25
Oct 31 03:56:54.538928 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.538900    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:56:54.679855 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.679822    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume\") pod \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") "
Oct 31 03:56:54.680069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.680051    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q\" (UniqueName: \"kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q\") pod \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\" (UID: \"ab34ecaf-1e52-4fc0-97a9-501eece79c03\") "
Oct 31 03:56:54.680301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.680278    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume" pod "ab34ecaf-1e52-4fc0-97a9-501eece79c03" (UID: "ab34ecaf-1e52-4fc0-97a9-501eece79c03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:56:54.680888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.680867    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab34ecaf-1e52-4fc0-97a9-501eece79c03-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:56:54.682333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.682312    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q" pod "ab34ecaf-1e52-4fc0-97a9-501eece79c03" (UID: "ab34ecaf-1e52-4fc0-97a9-501eece79c03"). InnerVolumeSpecName "kube-api-access-p4j6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:56:54.782990 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:54.782901    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4j6q\" (UniqueName: \"kubernetes.io/projected/ab34ecaf-1e52-4fc0-97a9-501eece79c03-kube-api-access-p4j6q\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:56:55.114468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.114342    5906 generic.go:356] "Generic (PLEG): container finished" podID="ab34ecaf-1e52-4fc0-97a9-501eece79c03" containerID="1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8" exitCode=0
Oct 31 03:56:55.114468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.114413    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"
Oct 31 03:56:55.114468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.114443    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76" event={"ID":"ab34ecaf-1e52-4fc0-97a9-501eece79c03","Type":"ContainerDied","Data":"1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8"}
Oct 31 03:56:55.114778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.114516    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76" event={"ID":"ab34ecaf-1e52-4fc0-97a9-501eece79c03","Type":"ContainerDied","Data":"1df0540410904662a19d68254cc6cb1ef6beb86dfda109d67ec9ba7a1951814e"}
Oct 31 03:56:55.114778 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.114542    5906 scope.go:122] "RemoveContainer" containerID="1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8"
Oct 31 03:56:55.127009 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.126985    5906 scope.go:122] "RemoveContainer" containerID="1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8"
Oct 31 03:56:55.127418 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:56:55.127384    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8\": container with ID starting with 1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8 not found: ID does not exist" containerID="1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8"
Oct 31 03:56:55.127517 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.127423    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8"} err="failed to get container status \"1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8\": rpc error: code = NotFound desc = could not find container \"1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8\": container with ID starting with 1945c4c947625655eb613efb3a066a43483e65d7901d61f170129ca05faaaae8 not found: ID does not exist"
Oct 31 03:56:55.152432 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.152400    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:56:55.154263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.154242    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:56:55.427096 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.426996    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"]
Oct 31 03:56:55.431212 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:55.431174    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-34a72a1f-797fbf87c5-6dw76"]
Oct 31 03:56:56.354069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:56.354033    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab34ecaf-1e52-4fc0-97a9-501eece79c03" path="/var/lib/kubelet/pods/ab34ecaf-1e52-4fc0-97a9-501eece79c03/volumes"
Oct 31 03:56:57.593490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:57.593454    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:56:59.561667 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:59.561629    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"]
Oct 31 03:56:59.688018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:59.687986    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"]
Oct 31 03:56:59.893770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:56:59.893668    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:00.107613 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:00.107574    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:02.090791 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:02.090732    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:04.289758 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:04.289723    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"]
Oct 31 03:57:04.399804 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:04.399763    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"]
Oct 31 03:57:04.562506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:04.562397    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv" podUID="490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" containerName="payload-processing" containerID="cri-o://ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780" gracePeriod=25
Oct 31 03:57:04.622565 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:04.622539    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:04.690647 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:04.689524    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h" podUID="8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" containerName="payload-pre-processing" containerID="cri-o://994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4" gracePeriod=25
Oct 31 03:57:04.798941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:04.782442    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:05.235026 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.234994    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:57:05.264342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.263881    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume\") pod \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") "
Oct 31 03:57:05.264342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.264130    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw\" (UniqueName: \"kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw\") pod \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\" (UID: \"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f\") "
Oct 31 03:57:05.264342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.264262    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume" pod "490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" (UID: "490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:05.264803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.264784    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:05.266275 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.266249    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw" pod "490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" (UID: "490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f"). InnerVolumeSpecName "kube-api-access-72mnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:05.276044 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.276015    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:57:05.295998 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.295963    5906 generic.go:356] "Generic (PLEG): container finished" podID="490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" containerID="ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780" exitCode=0
Oct 31 03:57:05.296607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.296013    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"
Oct 31 03:57:05.296607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.296045    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv" event={"ID":"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f","Type":"ContainerDied","Data":"ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780"}
Oct 31 03:57:05.296607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.296100    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv" event={"ID":"490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f","Type":"ContainerDied","Data":"9ac90bff2f4ad1ffa722693b0bcad523735e0872276785c4dea989410fc20637"}
Oct 31 03:57:05.296607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.296128    5906 scope.go:122] "RemoveContainer" containerID="ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780"
Oct 31 03:57:05.297741 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.297713    5906 generic.go:356] "Generic (PLEG): container finished" podID="8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" containerID="994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4" exitCode=0
Oct 31 03:57:05.297868 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.297750    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h" event={"ID":"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e","Type":"ContainerDied","Data":"994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4"}
Oct 31 03:57:05.297868 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.297769    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"
Oct 31 03:57:05.297868 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.297803    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h" event={"ID":"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e","Type":"ContainerDied","Data":"8530c2c000a1f4f001af76efa14467c501f08dbea103869fbac4613220af2270"}
Oct 31 03:57:05.307601 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.307583    5906 scope.go:122] "RemoveContainer" containerID="ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780"
Oct 31 03:57:05.307866 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:05.307848    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780\": container with ID starting with ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780 not found: ID does not exist" containerID="ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780"
Oct 31 03:57:05.307906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.307878    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780"} err="failed to get container status \"ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780\": rpc error: code = NotFound desc = could not find container \"ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780\": container with ID starting with ff86561180af55bcf47061a5b3dbb9b6bfb80640584640a36c5304c269d79780 not found: ID does not exist"
Oct 31 03:57:05.307906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.307896    5906 scope.go:122] "RemoveContainer" containerID="994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4"
Oct 31 03:57:05.317900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.317882    5906 scope.go:122] "RemoveContainer" containerID="994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4"
Oct 31 03:57:05.318187 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:05.318171    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4\": container with ID starting with 994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4 not found: ID does not exist" containerID="994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4"
Oct 31 03:57:05.318254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.318195    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4"} err="failed to get container status \"994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4\": rpc error: code = NotFound desc = could not find container \"994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4\": container with ID starting with 994d530a2a7f4cb1e57daa753a375fb2f5c384ac10b515e3165be2a0d0327ec4 not found: ID does not exist"
Oct 31 03:57:05.366135 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.366057    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume\") pod \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") "
Oct 31 03:57:05.366439 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.366422    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz\" (UniqueName: \"kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz\") pod \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\" (UID: \"8d5e93eb-f9d0-45a9-8924-35abafd0ce9e\") "
Oct 31 03:57:05.366489 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.366454    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume" pod "8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" (UID: "8d5e93eb-f9d0-45a9-8924-35abafd0ce9e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:05.367521 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.367495    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72mnw\" (UniqueName: \"kubernetes.io/projected/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f-kube-api-access-72mnw\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:05.367521 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.367523    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:05.369286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.369260    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz" pod "8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" (UID: "8d5e93eb-f9d0-45a9-8924-35abafd0ce9e"). InnerVolumeSpecName "kube-api-access-fzntz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:05.469454 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.469415    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzntz\" (UniqueName: \"kubernetes.io/projected/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e-kube-api-access-fzntz\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:05.473237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.473189    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"]
Oct 31 03:57:05.477048 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:05.477024    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-984fb1f2-8679b795f-bj9jv"]
Oct 31 03:57:06.009230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:06.008429    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"]
Oct 31 03:57:06.109227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:06.104648    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-984fb1f2-9b88b9687-9482h"]
Oct 31 03:57:06.356018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:06.355922    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" path="/var/lib/kubelet/pods/490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f/volumes"
Oct 31 03:57:06.356641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:06.356615    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" path="/var/lib/kubelet/pods/8d5e93eb-f9d0-45a9-8924-35abafd0ce9e/volumes"
Oct 31 03:57:06.384931 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:06.384891    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:57:06.590363 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:06.590320    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:09.291075 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.291026    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn" podUID="bab59f81-9832-4aa6-9956-3320c7dbed72" containerName="payload-processing" containerID="cri-o://623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94" gracePeriod=25
Oct 31 03:57:09.400089 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.400042    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr" podUID="3b629681-070f-4eb6-93a4-95467c37643e" containerName="payload-pre-processing" containerID="cri-o://87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6" gracePeriod=25
Oct 31 03:57:09.599571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.599175    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:57:09.656127 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.656082    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6\" (UniqueName: \"kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6\") pod \"bab59f81-9832-4aa6-9956-3320c7dbed72\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") "
Oct 31 03:57:09.656380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.656154    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume\") pod \"bab59f81-9832-4aa6-9956-3320c7dbed72\" (UID: \"bab59f81-9832-4aa6-9956-3320c7dbed72\") "
Oct 31 03:57:09.656657 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.656626    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume" pod "bab59f81-9832-4aa6-9956-3320c7dbed72" (UID: "bab59f81-9832-4aa6-9956-3320c7dbed72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:09.657235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.657195    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bab59f81-9832-4aa6-9956-3320c7dbed72-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:09.659900 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.659864    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6" pod "bab59f81-9832-4aa6-9956-3320c7dbed72" (UID: "bab59f81-9832-4aa6-9956-3320c7dbed72"). InnerVolumeSpecName "kube-api-access-n4lw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:09.715301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.715255    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:57:09.758376 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.758336    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume\") pod \"3b629681-070f-4eb6-93a4-95467c37643e\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") "
Oct 31 03:57:09.758583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.758565    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr\" (UniqueName: \"kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr\") pod \"3b629681-070f-4eb6-93a4-95467c37643e\" (UID: \"3b629681-070f-4eb6-93a4-95467c37643e\") "
Oct 31 03:57:09.758749 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.758723    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume" pod "3b629681-070f-4eb6-93a4-95467c37643e" (UID: "3b629681-070f-4eb6-93a4-95467c37643e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:09.759248 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.759226    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b629681-070f-4eb6-93a4-95467c37643e-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:09.759248 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.759249    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4lw6\" (UniqueName: \"kubernetes.io/projected/bab59f81-9832-4aa6-9956-3320c7dbed72-kube-api-access-n4lw6\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:09.760876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.760847    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr" pod "3b629681-070f-4eb6-93a4-95467c37643e" (UID: "3b629681-070f-4eb6-93a4-95467c37643e"). InnerVolumeSpecName "kube-api-access-46vlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:09.861190 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:09.861087    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46vlr\" (UniqueName: \"kubernetes.io/projected/3b629681-070f-4eb6-93a4-95467c37643e-kube-api-access-46vlr\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:10.353573 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.353538    5906 generic.go:356] "Generic (PLEG): container finished" podID="bab59f81-9832-4aa6-9956-3320c7dbed72" containerID="623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94" exitCode=0
Oct 31 03:57:10.354006 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.353645    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"
Oct 31 03:57:10.355080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355053    5906 generic.go:356] "Generic (PLEG): container finished" podID="3b629681-070f-4eb6-93a4-95467c37643e" containerID="87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6" exitCode=0
Oct 31 03:57:10.355217 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355147    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"
Oct 31 03:57:10.355801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355777    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn" event={"ID":"bab59f81-9832-4aa6-9956-3320c7dbed72","Type":"ContainerDied","Data":"623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94"}
Oct 31 03:57:10.355907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355826    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn" event={"ID":"bab59f81-9832-4aa6-9956-3320c7dbed72","Type":"ContainerDied","Data":"8b34b68372d326d2480bcf6ae9b902c26e541e17ca667bc1b1c9ad64af14ada1"}
Oct 31 03:57:10.355907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355851    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr" event={"ID":"3b629681-070f-4eb6-93a4-95467c37643e","Type":"ContainerDied","Data":"87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6"}
Oct 31 03:57:10.355907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355846    5906 scope.go:122] "RemoveContainer" containerID="623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94"
Oct 31 03:57:10.355907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.355877    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr" event={"ID":"3b629681-070f-4eb6-93a4-95467c37643e","Type":"ContainerDied","Data":"d8b1d3c640611a3a0d523f4c41425af2b3d0f4b503c7ef2c363c4358e5ba9a22"}
Oct 31 03:57:10.371489 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.371421    5906 scope.go:122] "RemoveContainer" containerID="623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94"
Oct 31 03:57:10.371947 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:10.371919    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94\": container with ID starting with 623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94 not found: ID does not exist" containerID="623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94"
Oct 31 03:57:10.372048 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.371959    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94"} err="failed to get container status \"623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94\": rpc error: code = NotFound desc = could not find container \"623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94\": container with ID starting with 623ed4bfbe1c0f7c0bb65da2b15f55a5365f831872e260762655a6fd04574d94 not found: ID does not exist"
Oct 31 03:57:10.372048 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.371984    5906 scope.go:122] "RemoveContainer" containerID="87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6"
Oct 31 03:57:10.381880 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.381862    5906 scope.go:122] "RemoveContainer" containerID="87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6"
Oct 31 03:57:10.382161 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:10.382132    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6\": container with ID starting with 87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6 not found: ID does not exist" containerID="87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6"
Oct 31 03:57:10.382292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.382161    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6"} err="failed to get container status \"87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6\": rpc error: code = NotFound desc = could not find container \"87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6\": container with ID starting with 87f4fe0538ddaf996f01a84d96b53d0ffd6bf96183046afac4d18601b4c7eea6 not found: ID does not exist"
Oct 31 03:57:10.527286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.527247    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"]
Oct 31 03:57:10.576308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.576227    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-af5cff2a-689cb6f77f-4mwlr"]
Oct 31 03:57:10.642367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.642335    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"]
Oct 31 03:57:10.647701 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:10.647671    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-af5cff2a-57868d4bf9-jzdnn"]
Oct 31 03:57:11.096553 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:11.096512    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:12.353697 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:12.353667    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b629681-070f-4eb6-93a4-95467c37643e" path="/var/lib/kubelet/pods/3b629681-070f-4eb6-93a4-95467c37643e/volumes"
Oct 31 03:57:12.354187 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:12.354172    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bab59f81-9832-4aa6-9956-3320c7dbed72" path="/var/lib/kubelet/pods/bab59f81-9832-4aa6-9956-3320c7dbed72/volumes"
Oct 31 03:57:13.300263 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:13.300223    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:15.501087 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:15.501049    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:15.654231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:15.638173    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:19.328493 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.328459    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"]
Oct 31 03:57:19.328917 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.328730    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" podUID="3b54e29e-b8c6-458b-a983-afa7824856bf" containerName="maas-api" containerID="cri-o://b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da" gracePeriod=30
Oct 31 03:57:19.422731 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.422694    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:19.786253 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.781639    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:57:19.870067 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.870030    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca\") pod \"3b54e29e-b8c6-458b-a983-afa7824856bf\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") "
Oct 31 03:57:19.871933 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.871875    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d\" (UniqueName: \"kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d\") pod \"3b54e29e-b8c6-458b-a983-afa7824856bf\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") "
Oct 31 03:57:19.872357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.872334    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs\") pod \"3b54e29e-b8c6-458b-a983-afa7824856bf\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") "
Oct 31 03:57:19.872572 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.872526    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\") pod \"3b54e29e-b8c6-458b-a983-afa7824856bf\" (UID: \"3b54e29e-b8c6-458b-a983-afa7824856bf\") "
Oct 31 03:57:19.875716 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.875668    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca" pod "3b54e29e-b8c6-458b-a983-afa7824856bf" (UID: "3b54e29e-b8c6-458b-a983-afa7824856bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:19.892545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.892474    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d" pod "3b54e29e-b8c6-458b-a983-afa7824856bf" (UID: "3b54e29e-b8c6-458b-a983-afa7824856bf"). InnerVolumeSpecName "kube-api-access-glc9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:19.892887 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.892837    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls" pod "3b54e29e-b8c6-458b-a983-afa7824856bf" (UID: "3b54e29e-b8c6-458b-a983-afa7824856bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:57:19.893178 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.893142    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs" pod "3b54e29e-b8c6-458b-a983-afa7824856bf" (UID: "3b54e29e-b8c6-458b-a983-afa7824856bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:57:19.978395 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.978297    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3b54e29e-b8c6-458b-a983-afa7824856bf-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:19.979745 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.979703    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glc9d\" (UniqueName: \"kubernetes.io/projected/3b54e29e-b8c6-458b-a983-afa7824856bf-kube-api-access-glc9d\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:19.979934 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.979919    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:19.980029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:19.980018    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3b54e29e-b8c6-458b-a983-afa7824856bf-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:20.426005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.425970    5906 generic.go:356] "Generic (PLEG): container finished" podID="3b54e29e-b8c6-458b-a983-afa7824856bf" containerID="b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da" exitCode=0
Oct 31 03:57:20.426005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.426016    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"
Oct 31 03:57:20.426574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.426058    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" event={"ID":"3b54e29e-b8c6-458b-a983-afa7824856bf","Type":"ContainerDied","Data":"b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da"}
Oct 31 03:57:20.426574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.426112    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz" event={"ID":"3b54e29e-b8c6-458b-a983-afa7824856bf","Type":"ContainerDied","Data":"e21f5f7ca920853dbc5a3fcfa3a8da1013dfb89ce7efcf422547b7a34e0e3f99"}
Oct 31 03:57:20.426574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.426134    5906 scope.go:122] "RemoveContainer" containerID="b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da"
Oct 31 03:57:20.441167 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.441140    5906 scope.go:122] "RemoveContainer" containerID="b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da"
Oct 31 03:57:20.441564 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:20.441542    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da\": container with ID starting with b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da not found: ID does not exist" containerID="b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da"
Oct 31 03:57:20.441639 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.441579    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da"} err="failed to get container status \"b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da\": rpc error: code = NotFound desc = could not find container \"b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da\": container with ID starting with b93e6be54a4a3207f8768be40a0fd2f5fe05bfde18a277138b7243627832b5da not found: ID does not exist"
Oct 31 03:57:20.598260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.598228    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"]
Oct 31 03:57:20.649982 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.649913    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-34a72a1f-95b8b8c8c-c5zgz"]
Oct 31 03:57:20.740453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.740420    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:20.980166 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:20.980072    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:21.156379 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:21.147765    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:21.566647 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:21.566609    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"]
Oct 31 03:57:21.567059 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:21.566881    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy" containerID="cri-o://3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41" gracePeriod=30
Oct 31 03:57:21.572072 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:21.572036    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:22.233051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:22.233007    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:22.354220 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:22.354175    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b54e29e-b8c6-458b-a983-afa7824856bf" path="/var/lib/kubelet/pods/3b54e29e-b8c6-458b-a983-afa7824856bf/volumes"
Oct 31 03:57:22.390200 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:22.390158    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:23.382595 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:23.382557    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:25.136797 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.136762    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"]
Oct 31 03:57:25.137305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.136973    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" podUID="07f23535-407b-46c4-90cd-01979f4302b1" containerName="maas-api" containerID="cri-o://6ae330bf4d953bdfb6011c043d1ec16ead55489f9df8a283699d9b632d59448a" gracePeriod=30
Oct 31 03:57:25.224391 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.224347    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:25.523705 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.523573    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:25.545352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.541730    5906 generic.go:356] "Generic (PLEG): container finished" podID="07f23535-407b-46c4-90cd-01979f4302b1" containerID="6ae330bf4d953bdfb6011c043d1ec16ead55489f9df8a283699d9b632d59448a" exitCode=0
Oct 31 03:57:25.545352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.541808    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" event={"ID":"07f23535-407b-46c4-90cd-01979f4302b1","Type":"ContainerDied","Data":"6ae330bf4d953bdfb6011c043d1ec16ead55489f9df8a283699d9b632d59448a"}
Oct 31 03:57:25.606286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.602467    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:57:25.642373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.642334    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs\") pod \"07f23535-407b-46c4-90cd-01979f4302b1\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") "
Oct 31 03:57:25.642760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.642731    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca\") pod \"07f23535-407b-46c4-90cd-01979f4302b1\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") "
Oct 31 03:57:25.643042 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.643021    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\") pod \"07f23535-407b-46c4-90cd-01979f4302b1\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") "
Oct 31 03:57:25.643442 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.643423    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst\" (UniqueName: \"kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst\") pod \"07f23535-407b-46c4-90cd-01979f4302b1\" (UID: \"07f23535-407b-46c4-90cd-01979f4302b1\") "
Oct 31 03:57:25.646943 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.646914    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca" pod "07f23535-407b-46c4-90cd-01979f4302b1" (UID: "07f23535-407b-46c4-90cd-01979f4302b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:25.650577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.650547    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst" pod "07f23535-407b-46c4-90cd-01979f4302b1" (UID: "07f23535-407b-46c4-90cd-01979f4302b1"). InnerVolumeSpecName "kube-api-access-7hhst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:25.707437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.705608    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls" pod "07f23535-407b-46c4-90cd-01979f4302b1" (UID: "07f23535-407b-46c4-90cd-01979f4302b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:57:25.707758 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.707704    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs" pod "07f23535-407b-46c4-90cd-01979f4302b1" (UID: "07f23535-407b-46c4-90cd-01979f4302b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:57:25.750077 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.750037    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07f23535-407b-46c4-90cd-01979f4302b1-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:25.750354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.750332    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:25.750483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.750468    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hhst\" (UniqueName: \"kubernetes.io/projected/07f23535-407b-46c4-90cd-01979f4302b1-kube-api-access-7hhst\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:25.750607 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:25.750574    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07f23535-407b-46c4-90cd-01979f4302b1-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:26.245233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.236296    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"]
Oct 31 03:57:26.247231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.247174    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" podUID="be93559a-bd9d-47c2-a3e4-55377c6deaaa" containerName="maas-api" containerID="cri-o://dd777586030b7388737fe4b83b4a0a9baac4e74821d5db85507623df8495ceaf" gracePeriod=30
Oct 31 03:57:26.308668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.308504    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:26.665592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.650850    5906 generic.go:356] "Generic (PLEG): container finished" podID="be93559a-bd9d-47c2-a3e4-55377c6deaaa" containerID="dd777586030b7388737fe4b83b4a0a9baac4e74821d5db85507623df8495ceaf" exitCode=0
Oct 31 03:57:26.665592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.651040    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" event={"ID":"be93559a-bd9d-47c2-a3e4-55377c6deaaa","Type":"ContainerDied","Data":"dd777586030b7388737fe4b83b4a0a9baac4e74821d5db85507623df8495ceaf"}
Oct 31 03:57:26.665592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.651769    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:26.665592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.658655    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh" event={"ID":"07f23535-407b-46c4-90cd-01979f4302b1","Type":"ContainerDied","Data":"364886a25000338563c4612ef693ac810b28d39ecc5ef029b10698019b5c52f7"}
Oct 31 03:57:26.665592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.658744    5906 scope.go:122] "RemoveContainer" containerID="6ae330bf4d953bdfb6011c043d1ec16ead55489f9df8a283699d9b632d59448a"
Oct 31 03:57:26.665592 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.658962    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"
Oct 31 03:57:26.926225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.918314    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:57:26.926225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.918602    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"]
Oct 31 03:57:26.926225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.920116    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:26.926225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.924193    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-984fb1f2-8558659475-vl2zh"]
Oct 31 03:57:26.946939 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.946901    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\") pod \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") "
Oct 31 03:57:26.947119 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.947062    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca\") pod \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") "
Oct 31 03:57:26.947119 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.947100    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs\") pod \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") "
Oct 31 03:57:26.947196 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.947187    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72\" (UniqueName: \"kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72\") pod \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\" (UID: \"be93559a-bd9d-47c2-a3e4-55377c6deaaa\") "
Oct 31 03:57:26.947621 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.947596    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca" pod "be93559a-bd9d-47c2-a3e4-55377c6deaaa" (UID: "be93559a-bd9d-47c2-a3e4-55377c6deaaa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:26.947968 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.947945    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be93559a-bd9d-47c2-a3e4-55377c6deaaa-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:26.949810 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.949789    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls" pod "be93559a-bd9d-47c2-a3e4-55377c6deaaa" (UID: "be93559a-bd9d-47c2-a3e4-55377c6deaaa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:57:26.950877 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.950853    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs" pod "be93559a-bd9d-47c2-a3e4-55377c6deaaa" (UID: "be93559a-bd9d-47c2-a3e4-55377c6deaaa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:57:26.951193 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:26.951167    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72" pod "be93559a-bd9d-47c2-a3e4-55377c6deaaa" (UID: "be93559a-bd9d-47c2-a3e4-55377c6deaaa"). InnerVolumeSpecName "kube-api-access-xwl72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:27.035095 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.035012    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:57:27.049199 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049162    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049302    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049385    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049428 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049420    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049468    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049483    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:27.049599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049496    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049755 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049649    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049878 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049857    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.049945 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.049900    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket\") pod \"3e29db75-d021-42f2-b9d0-49bc35b99864\" (UID: \"3e29db75-d021-42f2-b9d0-49bc35b99864\") "
Oct 31 03:57:27.050035 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.050010    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:27.050888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.050859    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.051188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.050894    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be93559a-bd9d-47c2-a3e4-55377c6deaaa-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.051188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.050910    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwl72\" (UniqueName: \"kubernetes.io/projected/be93559a-bd9d-47c2-a3e4-55377c6deaaa-kube-api-access-xwl72\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.051188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.050925    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.051188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.050939    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.051188 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.051151    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:27.051554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.051530    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:27.051801 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.051775    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:27.052284 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.052259    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:27.052597 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.052570    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "kube-api-access-9zlnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:27.052694 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.052617    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:27.054445 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.054418    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo" pod "3e29db75-d021-42f2-b9d0-49bc35b99864" (UID: "3e29db75-d021-42f2-b9d0-49bc35b99864"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:57:27.061360 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.061171    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153863    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zlnx\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-kube-api-access-9zlnx\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153910    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153930    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153944    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e29db75-d021-42f2-b9d0-49bc35b99864-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153958    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153972    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e29db75-d021-42f2-b9d0-49bc35b99864-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.154071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.153985    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e29db75-d021-42f2-b9d0-49bc35b99864-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:27.667747 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.667709    5906 generic.go:356] "Generic (PLEG): container finished" podID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerID="3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41" exitCode=0
Oct 31 03:57:27.668295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.667767    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"
Oct 31 03:57:27.668295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.667793    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" event={"ID":"3e29db75-d021-42f2-b9d0-49bc35b99864","Type":"ContainerDied","Data":"3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41"}
Oct 31 03:57:27.668295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.667860    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls" event={"ID":"3e29db75-d021-42f2-b9d0-49bc35b99864","Type":"ContainerDied","Data":"566447972dae76c0fc001222c3e239a72236d20504c32cc705d23edad8b555fb"}
Oct 31 03:57:27.668295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.667892    5906 scope.go:122] "RemoveContainer" containerID="3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41"
Oct 31 03:57:27.669556 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.669532    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"
Oct 31 03:57:27.669679 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.669582    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz" event={"ID":"be93559a-bd9d-47c2-a3e4-55377c6deaaa","Type":"ContainerDied","Data":"59f5bcc814be2d45286f6440fc4df1337ac1a92d6afd42a583b80c07e283bf5c"}
Oct 31 03:57:27.678455 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.678440    5906 scope.go:122] "RemoveContainer" containerID="154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"
Oct 31 03:57:27.690739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.690598    5906 scope.go:122] "RemoveContainer" containerID="3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41"
Oct 31 03:57:27.691111 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:27.691089    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41\": container with ID starting with 3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41 not found: ID does not exist" containerID="3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41"
Oct 31 03:57:27.691213 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.691119    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41"} err="failed to get container status \"3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41\": rpc error: code = NotFound desc = could not find container \"3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41\": container with ID starting with 3a5c52b8f82176fe148ecb2d150ae1aca6eb842570dfa72db8cfb9e236ea5e41 not found: ID does not exist"
Oct 31 03:57:27.691213 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.691138    5906 scope.go:122] "RemoveContainer" containerID="154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"
Oct 31 03:57:27.691450 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:27.691431    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82\": container with ID starting with 154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82 not found: ID does not exist" containerID="154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"
Oct 31 03:57:27.691496 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.691457    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82"} err="failed to get container status \"154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82\": rpc error: code = NotFound desc = could not find container \"154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82\": container with ID starting with 154b041ffd1929654f93f42312ed01792052bfae3a08c0c396b28bf78b748f82 not found: ID does not exist"
Oct 31 03:57:27.691496 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.691474    5906 scope.go:122] "RemoveContainer" containerID="dd777586030b7388737fe4b83b4a0a9baac4e74821d5db85507623df8495ceaf"
Oct 31 03:57:27.772084 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.772049    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:27.775604 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.775575    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"]
Oct 31 03:57:27.775788 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.775769    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy" containerID="cri-o://4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3" gracePeriod=30
Oct 31 03:57:27.932466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.932180    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:27.947508 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.947470    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"]
Oct 31 03:57:27.964252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:27.956252    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af5cff2a-84c88db7ff-8l6zz"]
Oct 31 03:57:28.047225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.042694    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"]
Oct 31 03:57:28.067239 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.066774    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-34a72a1f-gw-openshift-default-7f5685b4d9-qpjls"]
Oct 31 03:57:28.108231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.104891    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:28.266290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.245860    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:28.359251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.359115    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07f23535-407b-46c4-90cd-01979f4302b1" path="/var/lib/kubelet/pods/07f23535-407b-46c4-90cd-01979f4302b1/volumes"
Oct 31 03:57:28.359952 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.359933    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" path="/var/lib/kubelet/pods/3e29db75-d021-42f2-b9d0-49bc35b99864/volumes"
Oct 31 03:57:28.360600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.360582    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be93559a-bd9d-47c2-a3e4-55377c6deaaa" path="/var/lib/kubelet/pods/be93559a-bd9d-47c2-a3e4-55377c6deaaa/volumes"
Oct 31 03:57:28.438835 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.438799    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"]
Oct 31 03:57:28.439128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.439095    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy" containerID="cri-o://2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3" gracePeriod=30
Oct 31 03:57:28.457745 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:28.457708    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:29.190524 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.190488    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:29.448817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.448733    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:29.691064 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691023    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"]
Oct 31 03:57:29.691739 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691724    5906 cpu_manager.go:414] "containerMap: removing container" podUID="07f23535-407b-46c4-90cd-01979f4302b1" containerName="maas-api"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691740    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07f23535-407b-46c4-90cd-01979f4302b1" containerName="maas-api"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691752    5906 cpu_manager.go:414] "containerMap: removing container" podUID="be93559a-bd9d-47c2-a3e4-55377c6deaaa" containerName="maas-api"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691758    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be93559a-bd9d-47c2-a3e4-55377c6deaaa" containerName="maas-api"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691770    5906 cpu_manager.go:414] "containerMap: removing container" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691775    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691794    5906 cpu_manager.go:414] "containerMap: removing container" podUID="bab59f81-9832-4aa6-9956-3320c7dbed72" containerName="payload-processing"
Oct 31 03:57:29.691803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691799    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bab59f81-9832-4aa6-9956-3320c7dbed72" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691808    5906 cpu_manager.go:414] "containerMap: removing container" podUID="490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691813    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691818    5906 cpu_manager.go:414] "containerMap: removing container" podUID="3b629681-070f-4eb6-93a4-95467c37643e" containerName="payload-pre-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691823    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b629681-070f-4eb6-93a4-95467c37643e" containerName="payload-pre-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691839    5906 cpu_manager.go:414] "containerMap: removing container" podUID="ab34ecaf-1e52-4fc0-97a9-501eece79c03" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691844    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab34ecaf-1e52-4fc0-97a9-501eece79c03" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691854    5906 cpu_manager.go:414] "containerMap: removing container" podUID="3b54e29e-b8c6-458b-a983-afa7824856bf" containerName="maas-api"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691859    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b54e29e-b8c6-458b-a983-afa7824856bf" containerName="maas-api"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691868    5906 cpu_manager.go:414] "containerMap: removing container" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691872    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691879    5906 cpu_manager.go:414] "containerMap: removing container" podUID="8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" containerName="payload-pre-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691883    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" containerName="payload-pre-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691973    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="bab59f81-9832-4aa6-9956-3320c7dbed72" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691980    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="490f9cd4-26c0-48b2-8ff5-9a8fecc5ec8f" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691986    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b629681-070f-4eb6-93a4-95467c37643e" containerName="payload-pre-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691993    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab34ecaf-1e52-4fc0-97a9-501eece79c03" containerName="payload-processing"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.691998    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b54e29e-b8c6-458b-a983-afa7824856bf" containerName="maas-api"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.692006    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy"
Oct 31 03:57:29.692005 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.692014    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d5e93eb-f9d0-45a9-8924-35abafd0ce9e" containerName="payload-pre-processing"
Oct 31 03:57:29.692568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.692025    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="07f23535-407b-46c4-90cd-01979f4302b1" containerName="maas-api"
Oct 31 03:57:29.692568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.692032    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="be93559a-bd9d-47c2-a3e4-55377c6deaaa" containerName="maas-api"
Oct 31 03:57:29.692568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.692039    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e29db75-d021-42f2-b9d0-49bc35b99864" containerName="istio-proxy"
Oct 31 03:57:29.695757 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.695737    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.839649    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.839712    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh6mw\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.839760    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.839865    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.839966    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.840075    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.840124    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.840159    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.871226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.840298    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.923505 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.921820    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.998706    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.998782    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.998822    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.998922    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.999039    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.999085    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh6mw\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.999128    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.999245    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:29.999813 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.999336    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.022250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:29.999951    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.022250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.001932    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.022250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.004841    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w5-938507-openshift-default-dockercfg-tvc8t\""
Oct 31 03:57:30.022250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.007766    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.022250 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.008338    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.024061 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.024029    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"]
Oct 31 03:57:30.027190 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.025631    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.097317 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.094987    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.097317 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.095971    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.130231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.119818    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh6mw\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.130231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.121795    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token\") pod \"e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") " pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.311689 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.311591    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:30.540963 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.540528    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:30.617615 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.617511    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"]
Oct 31 03:57:30.682742 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.682583    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:30.717226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.687372    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"]
Oct 31 03:57:30.717226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.687503    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.794240 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.746515    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6c89303e-openshift-default-dockercfg-dkxsx\""
Oct 31 03:57:30.823106 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823063    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823452 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823424    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823531    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823620 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823566    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vs8f\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823678 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823640    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823683    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823777 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823750    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823833 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823778    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.823833 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.823816    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927311 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927219    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927527 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927335    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927527 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927367    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vs8f\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927527 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927444    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927527 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927484    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927553    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927584    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927623    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927666    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.927817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.927764    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.928070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.928011    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.928221 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.928186    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.928354 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.928318    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.928422 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.928378    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.930681 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.930656    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.930872 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.930855    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.991226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.991185    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:30.991385 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:30.991351    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vs8f\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f\") pod \"e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") " pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:31.026444 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.018425    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:31.129944 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.129812    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:31.163780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.163166    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:31.163780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.163276    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:31.163780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.163327    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:31.171347 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.171321    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"]
Oct 31 03:57:31.526026 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.525994    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"]
Oct 31 03:57:31.528600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.528552    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:31.528707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.528645    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:31.528707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:31.528685    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:32.101548 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.101508    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" event={"ID":"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b","Type":"ContainerStarted","Data":"708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"}
Oct 31 03:57:32.101743 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.101576    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" event={"ID":"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b","Type":"ContainerStarted","Data":"f49a7ec74b475fbcafb66dedd45c6802911eb9d272f5b0c1a6c2cd7121216a3d"}
Oct 31 03:57:32.103877 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.103849    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" event={"ID":"ae211c5c-03d3-4978-85ec-65f045d3d318","Type":"ContainerStarted","Data":"78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"}
Oct 31 03:57:32.104013 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.103903    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" event={"ID":"ae211c5c-03d3-4978-85ec-65f045d3d318","Type":"ContainerStarted","Data":"6f0ef310a44c5f95ed6d2af8a68b4fc74b855531c1f8ce4c259d87c25cd7d68c"}
Oct 31 03:57:32.130565 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.130528    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:32.135807 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.135779    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:32.229699 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.229644    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podStartSLOduration=2.229627317 podStartE2EDuration="2.229627317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:32.225553323 +0000 UTC m=+1702.059413658" watchObservedRunningTime="2026-10-31 03:57:32.229627317 +0000 UTC m=+1702.063487622"
Oct 31 03:57:32.313475 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.313433    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:32.319458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.319425    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:32.507551 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.507498    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podStartSLOduration=3.507483079 podStartE2EDuration="3.507483079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:32.506806546 +0000 UTC m=+1702.340666851" watchObservedRunningTime="2026-10-31 03:57:32.507483079 +0000 UTC m=+1702.341343384"
Oct 31 03:57:32.788269 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.788235    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"]
Oct 31 03:57:32.793029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.793006    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:32.890228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:32.872761    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-eb5ce91f-openshift-default-dockercfg-9mffj\""
Oct 31 03:57:33.212752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.145568    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"]
Oct 31 03:57:33.212752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.158818    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:33.237130 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.236850    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.237490 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.237466    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbzsj\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.246929 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.246889    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.247408 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.247387    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.247554 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.247539    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.248025 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.247986    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.248276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.248252    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.252306 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.252274    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.254108 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.254071    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.377684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.377640    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.421079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.421038    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbzsj\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.419025    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.416860    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438064    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438096    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438118    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438146    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438284    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438320    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438405    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.438687    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.439177    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.439884    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.440373 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.440027    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.441078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.440399    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.441078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.440659    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.441078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.440765    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.446256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.443587    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.446256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.445009    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.446256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.445712    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:33.499227 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.495793    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbzsj\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.524854 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.524749    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token\") pod \"e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") " pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.600232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.596257    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:33.634244 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.630893    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:33.807663 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.807452    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:57:33.869862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.869703    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.869862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.869805    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870000    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.870174 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870136    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870249    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870349 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870321    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870352    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870384    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870434    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.870555 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870513    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs\") pod \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\" (UID: \"f4ee9a10-bd94-44ac-b4d8-2038181b18d5\") "
Oct 31 03:57:33.871438 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.870874    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.871438 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.871092    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.871438 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.871331    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.871438 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.871395    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:33.872176 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.872158    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.872381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.872184    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.872381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.872200    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.872381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.872248    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.872381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.872263    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.873925 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.873898    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "kube-api-access-cjcvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:33.874079 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.873900    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:57:33.874244 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.874195    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:33.875693 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.875668    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy" pod "f4ee9a10-bd94-44ac-b4d8-2038181b18d5" (UID: "f4ee9a10-bd94-44ac-b4d8-2038181b18d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.897122 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.897089    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:57:33.973325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973275    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973488 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973391    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973593 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973573    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973640    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973646    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.973781 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973696    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973781 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973728    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973824    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973934 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973920    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.973994 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973917    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.974047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.973993    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy\") pod \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\" (UID: \"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f\") "
Oct 31 03:57:33.974047 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.974026    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.974279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.974256    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:33.974364 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.974323    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:57:33.974986 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.974957    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.974991    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975008    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjcvs\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-kube-api-access-cjcvs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975023    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975036    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975049    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4ee9a10-bd94-44ac-b4d8-2038181b18d5-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975070    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975085    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.975113 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.975099    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:33.976537 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.976505    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:33.976670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.976613    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "kube-api-access-5vgtl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:57:33.976670 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.976606    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:57:33.977364 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:33.977340    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy" pod "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" (UID: "b62bb728-1e8f-4a75-bca4-bdc12ddecb5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:57:34.077066 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.076974    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:34.077066 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.077022    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vgtl\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-kube-api-access-5vgtl\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:34.077066 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.077039    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:34.077066 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.077056    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:57:34.149728 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:34.149696    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2f33f4_6678_4542_a9df_0a2a3781c5ab.slice/crio-3506ceb33581dccac8d5405546b4f1cf151364228812ecb555debb071d085eb8 WatchSource:0}: Error finding container 3506ceb33581dccac8d5405546b4f1cf151364228812ecb555debb071d085eb8: Status 404 returned error can't find the container with id 3506ceb33581dccac8d5405546b4f1cf151364228812ecb555debb071d085eb8
Oct 31 03:57:34.152972 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.152926    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:34.153064 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.153021    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:34.153116 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.153066    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:34.153786 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.153765    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"]
Oct 31 03:57:34.266147 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.266110    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:34.420343 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.420303    5906 generic.go:356] "Generic (PLEG): container finished" podID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerID="2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3" exitCode=0
Oct 31 03:57:34.420518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.420368    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"
Oct 31 03:57:34.420518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.420388    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" event={"ID":"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f","Type":"ContainerDied","Data":"2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3"}
Oct 31 03:57:34.420518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.420458    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5" event={"ID":"b62bb728-1e8f-4a75-bca4-bdc12ddecb5f","Type":"ContainerDied","Data":"fd9a9900fb5d3d5f910d139eb8ac66a193eec99f7f567c60e2d4a2f8a196c30b"}
Oct 31 03:57:34.420518 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.420486    5906 scope.go:122] "RemoveContainer" containerID="2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3"
Oct 31 03:57:34.422920 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.422894    5906 generic.go:356] "Generic (PLEG): container finished" podID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerID="4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3" exitCode=0
Oct 31 03:57:34.423059 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.422969    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"
Oct 31 03:57:34.423059 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.422976    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" event={"ID":"f4ee9a10-bd94-44ac-b4d8-2038181b18d5","Type":"ContainerDied","Data":"4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3"}
Oct 31 03:57:34.423199 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.423059    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd" event={"ID":"f4ee9a10-bd94-44ac-b4d8-2038181b18d5","Type":"ContainerDied","Data":"5d35693ad422f6217f63817f8ba521648b94a2a835ebf1527df802bd8600ad68"}
Oct 31 03:57:34.425147 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.425119    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" event={"ID":"fc2f33f4-6678-4542-a9df-0a2a3781c5ab","Type":"ContainerStarted","Data":"3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"}
Oct 31 03:57:34.425316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.425169    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" event={"ID":"fc2f33f4-6678-4542-a9df-0a2a3781c5ab","Type":"ContainerStarted","Data":"3506ceb33581dccac8d5405546b4f1cf151364228812ecb555debb071d085eb8"}
Oct 31 03:57:34.448976 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.448938    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:34.461107 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.461079    5906 scope.go:122] "RemoveContainer" containerID="39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"
Oct 31 03:57:34.471797 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.471763    5906 scope.go:122] "RemoveContainer" containerID="2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3"
Oct 31 03:57:34.472126 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:34.472095    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3\": container with ID starting with 2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3 not found: ID does not exist" containerID="2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3"
Oct 31 03:57:34.472245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.472134    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3"} err="failed to get container status \"2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3\": rpc error: code = NotFound desc = could not find container \"2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3\": container with ID starting with 2e13997da3f44061ea55fc579e6b918ab000121de7da0b35542e32ed10396ed3 not found: ID does not exist"
Oct 31 03:57:34.472245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.472154    5906 scope.go:122] "RemoveContainer" containerID="39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"
Oct 31 03:57:34.472446 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:34.472427    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c\": container with ID starting with 39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c not found: ID does not exist" containerID="39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"
Oct 31 03:57:34.472503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.472451    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c"} err="failed to get container status \"39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c\": rpc error: code = NotFound desc = could not find container \"39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c\": container with ID starting with 39e0de6bf354cd8ec431f7a5792d97d6f8a1013e2387a9a44c7f92f21978bd4c not found: ID does not exist"
Oct 31 03:57:34.472503 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.472469    5906 scope.go:122] "RemoveContainer" containerID="4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3"
Oct 31 03:57:34.482834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.482806    5906 scope.go:122] "RemoveContainer" containerID="cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"
Oct 31 03:57:34.492929 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.492907    5906 scope.go:122] "RemoveContainer" containerID="4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3"
Oct 31 03:57:34.493273 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:34.493250    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3\": container with ID starting with 4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3 not found: ID does not exist" containerID="4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3"
Oct 31 03:57:34.493370 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.493279    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3"} err="failed to get container status \"4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3\": rpc error: code = NotFound desc = could not find container \"4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3\": container with ID starting with 4a5ffcab5b7f612000ba9aa0ab4512122613c9985fd82eae9bf837e350530ca3 not found: ID does not exist"
Oct 31 03:57:34.493370 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.493298    5906 scope.go:122] "RemoveContainer" containerID="cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"
Oct 31 03:57:34.493596 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:34.493571    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9\": container with ID starting with cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9 not found: ID does not exist" containerID="cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"
Oct 31 03:57:34.493634 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.493607    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9"} err="failed to get container status \"cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9\": rpc error: code = NotFound desc = could not find container \"cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9\": container with ID starting with cfcc8f55bccc99eaee4cb850cefd22471cf018d98f90cd03c413f2726d0afad9 not found: ID does not exist"
Oct 31 03:57:34.586750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.586715    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:34.602785 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.602734    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podStartSLOduration=2.6027205589999998 podStartE2EDuration="2.602720559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:34.5996151 +0000 UTC m=+1704.433475416" watchObservedRunningTime="2026-10-31 03:57:34.602720559 +0000 UTC m=+1704.436580863"
Oct 31 03:57:34.632015 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.631919    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:34.723787 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.723756    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"]
Oct 31 03:57:34.736609 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.729823    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-af5cff2a-openshift-default-d876f75fc-nttp5"]
Oct 31 03:57:34.855266 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.852847    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"]
Oct 31 03:57:34.876909 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:34.876856    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-984fb1f2-openshift-default-69f8fdc77-4xvtd"]
Oct 31 03:57:35.638059 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:35.638030    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:36.354276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:36.354245    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" path="/var/lib/kubelet/pods/b62bb728-1e8f-4a75-bca4-bdc12ddecb5f/volumes"
Oct 31 03:57:36.354817 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:36.354803    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" path="/var/lib/kubelet/pods/f4ee9a10-bd94-44ac-b4d8-2038181b18d5/volumes"
Oct 31 03:57:36.382947 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:57:36.382909    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:57:36.441134 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:36.441104    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:36.442315 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:36.442291    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:57:36.817074 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:36.817040    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:36.999752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:36.999717    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:37.215128 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.215030    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:37.381927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.381895    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:37.592596 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.592558    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:37.803405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.803372    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"]
Oct 31 03:57:37.804074 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804060    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy"
Oct 31 03:57:37.804121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804075    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy"
Oct 31 03:57:37.804121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804093    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy"
Oct 31 03:57:37.804121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804098    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy"
Oct 31 03:57:37.804121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804106    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy"
Oct 31 03:57:37.804121 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804112    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy"
Oct 31 03:57:37.804325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804124    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy"
Oct 31 03:57:37.804325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804129    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy"
Oct 31 03:57:37.804325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804238    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy"
Oct 31 03:57:37.804325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804249    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b62bb728-1e8f-4a75-bca4-bdc12ddecb5f" containerName="istio-proxy"
Oct 31 03:57:37.804325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804256    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy"
Oct 31 03:57:37.804325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.804263    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4ee9a10-bd94-44ac-b4d8-2038181b18d5" containerName="istio-proxy"
Oct 31 03:57:37.809097 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.809076    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:37.948139 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.912378    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"]
Oct 31 03:57:37.948733 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.934529    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-6c89303e\""
Oct 31 03:57:37.948891 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.934668    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-6c89303e-dockercfg-ncksg\""
Oct 31 03:57:37.969274 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.950229    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.007228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:37.970079    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmvm4\" (UniqueName: \"kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4\") pod \"payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.098656 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.098616    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"]
Oct 31 03:57:38.109138 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.104821    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.109138 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.109104    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmvm4\" (UniqueName: \"kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4\") pod \"payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.109404 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.109349    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.110636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.110390    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.118847 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.114254    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"]
Oct 31 03:57:38.230861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.220533    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt6jp\" (UniqueName: \"kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp\") pod \"payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") " pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.230861 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.220624    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume\") pod \"payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") " pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.238702 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.238624    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmvm4\" (UniqueName: \"kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4\") pod \"payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.250725 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.250684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:38.331802 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.330951    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt6jp\" (UniqueName: \"kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp\") pod \"payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") " pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.331802 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.331041    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume\") pod \"payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") " pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.332348 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.332287    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume\") pod \"payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") " pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.427192 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.427160    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:38.446612 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.446532    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:38.464495 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.464464    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt6jp\" (UniqueName: \"kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp\") pod \"payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") " pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.515974 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.515723    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:38.588017 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:38.587946    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:39.106318 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:39.106277    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd2b0cc8_7f41_4bf1_8fc0_a4bac900d7c6.slice/crio-7d0ca9aa7df8d1277188177c6746471f951b2a217423f6071ab04534ad575e49 WatchSource:0}: Error finding container 7d0ca9aa7df8d1277188177c6746471f951b2a217423f6071ab04534ad575e49: Status 404 returned error can't find the container with id 7d0ca9aa7df8d1277188177c6746471f951b2a217423f6071ab04534ad575e49
Oct 31 03:57:39.110312 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.110286    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"]
Oct 31 03:57:39.344190 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:39.344154    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6138a6_771e_4e1c_a54a_af370f42e346.slice/crio-aea4a99a51d25135900d8121681984a87a9c2f04946e457eb31f530fab68e2b6 WatchSource:0}: Error finding container aea4a99a51d25135900d8121681984a87a9c2f04946e457eb31f530fab68e2b6: Status 404 returned error can't find the container with id aea4a99a51d25135900d8121681984a87a9c2f04946e457eb31f530fab68e2b6
Oct 31 03:57:39.348316 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.348292    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"]
Oct 31 03:57:39.486142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.486099    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb" event={"ID":"cf6138a6-771e-4e1c-a54a-af370f42e346","Type":"ContainerStarted","Data":"c0ae1f548ea2cf3053d91c823db78e06ef3a465a727607763720fbed10d134db"}
Oct 31 03:57:39.486688 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.486176    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:39.486688 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.486216    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb" event={"ID":"cf6138a6-771e-4e1c-a54a-af370f42e346","Type":"ContainerStarted","Data":"aea4a99a51d25135900d8121681984a87a9c2f04946e457eb31f530fab68e2b6"}
Oct 31 03:57:39.488277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.488243    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm" event={"ID":"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6","Type":"ContainerStarted","Data":"44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b"}
Oct 31 03:57:39.488433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.488304    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm" event={"ID":"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6","Type":"ContainerStarted","Data":"7d0ca9aa7df8d1277188177c6746471f951b2a217423f6071ab04534ad575e49"}
Oct 31 03:57:39.488433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.488352    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:39.609219 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.609139    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb" podStartSLOduration=2.6091199769999998 podStartE2EDuration="2.609119977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:39.604402875 +0000 UTC m=+1709.438263179" watchObservedRunningTime="2026-10-31 03:57:39.609119977 +0000 UTC m=+1709.442980283"
Oct 31 03:57:39.782158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:39.782082    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm" podStartSLOduration=2.78205892 podStartE2EDuration="2.78205892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:39.752560685 +0000 UTC m=+1709.586421014" watchObservedRunningTime="2026-10-31 03:57:39.78205892 +0000 UTC m=+1709.615919226"
Oct 31 03:57:42.052062 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.052024    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:42.226228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.225569    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:42.540424 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.540388    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"]
Oct 31 03:57:42.620230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.585532    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:42.620230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.590929    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"]
Oct 31 03:57:42.704228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.695422    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:42.704228 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.695758    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l9zm\" (UniqueName: \"kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm\") pod \"payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:42.813461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.809521    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w5-938507\""
Oct 31 03:57:42.813461 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.810123    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w5-938507-dockercfg-b2k5h\""
Oct 31 03:57:42.867883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.861863    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l9zm\" (UniqueName: \"kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm\") pod \"payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:42.867883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.863802    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:42.867883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.866273    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:42.868486 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:42.868455    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"]
Oct 31 03:57:43.029458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.019932    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"]
Oct 31 03:57:43.029458 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.020069    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.081285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.080455    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l9zm\" (UniqueName: \"kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm\") pod \"payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:43.109386 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.109337    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:43.196473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.196442    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5zgg\" (UniqueName: \"kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg\") pod \"payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.196712 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.196681    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.298672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.298633    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5zgg\" (UniqueName: \"kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg\") pod \"payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.298873 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.298760    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.299529 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.299503    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.362020 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.361932    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5zgg\" (UniqueName: \"kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg\") pod \"payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.397162 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.397127    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.454898 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.453754    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:43.541267 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.538803    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"]
Oct 31 03:57:43.779867 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:43.779839    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57f0f9df_fecb_415f_8cac_441031401154.slice/crio-1e35c1c6028d800de7d54150029f54e2600c4c61d9d64e908eb11c29589312e5 WatchSource:0}: Error finding container 1e35c1c6028d800de7d54150029f54e2600c4c61d9d64e908eb11c29589312e5: Status 404 returned error can't find the container with id 1e35c1c6028d800de7d54150029f54e2600c4c61d9d64e908eb11c29589312e5
Oct 31 03:57:43.781525 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.781503    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"]
Oct 31 03:57:43.875167 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.875127    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx" event={"ID":"9a787113-91b9-4814-8828-501ac0bb5f76","Type":"ContainerStarted","Data":"6fbe651456b978fe270b11dc1c6156a0124228f55ddb4745ef5d7c2eb7412179"}
Oct 31 03:57:43.875390 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.875196    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:43.875390 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.875235    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx" event={"ID":"9a787113-91b9-4814-8828-501ac0bb5f76","Type":"ContainerStarted","Data":"51d7d6b5e65aeb5e6e5764ae2b317076b77712d5a48430c1e41d9a3b7bc69182"}
Oct 31 03:57:43.876776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.876701    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc" event={"ID":"57f0f9df-fecb-415f-8cac-441031401154","Type":"ContainerStarted","Data":"f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87"}
Oct 31 03:57:43.876776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.876742    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc" event={"ID":"57f0f9df-fecb-415f-8cac-441031401154","Type":"ContainerStarted","Data":"1e35c1c6028d800de7d54150029f54e2600c4c61d9d64e908eb11c29589312e5"}
Oct 31 03:57:43.876946 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.876790    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:43.996571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:43.996511    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx" podStartSLOduration=1.996495407 podStartE2EDuration="1.996495407s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:43.995887798 +0000 UTC m=+1713.829748104" watchObservedRunningTime="2026-10-31 03:57:43.996495407 +0000 UTC m=+1713.830355711"
Oct 31 03:57:44.515358 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:44.515302    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:44.767279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:44.745052    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:45.014469 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.014406    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc" podStartSLOduration=3.014381983 podStartE2EDuration="3.014381983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:44.123538472 +0000 UTC m=+1713.957398787" watchObservedRunningTime="2026-10-31 03:57:45.014381983 +0000 UTC m=+1714.848242288"
Oct 31 03:57:45.031277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.018254    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"]
Oct 31 03:57:45.045230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.038736    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.070050 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.070020    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"]
Oct 31 03:57:45.124712 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.124670    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"]
Oct 31 03:57:45.226370 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.153423    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-eb5ce91f-dockercfg-8bm76\""
Oct 31 03:57:45.226606 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.153524    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-eb5ce91f\""
Oct 31 03:57:45.252574 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.252517    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsgj2\" (UniqueName: \"kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2\") pod \"payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.252776 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.252621    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.271522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.271487    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"]
Oct 31 03:57:45.280083 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.271848    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.399774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.359314    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume\") pod \"payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") " pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.399774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.359441    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5ljd\" (UniqueName: \"kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd\") pod \"payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") " pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.399774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.359736    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsgj2\" (UniqueName: \"kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2\") pod \"payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.399774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.359827    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.399774 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.372779    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.476236 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.457189    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsgj2\" (UniqueName: \"kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2\") pod \"payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.532329 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.494713    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume\") pod \"payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") " pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.533108 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.533077    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5ljd\" (UniqueName: \"kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd\") pod \"payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") " pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.534437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.534414    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume\") pod \"payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") " pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.599390 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.598919    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5ljd\" (UniqueName: \"kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd\") pod \"payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") " pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:45.703051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.702964    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:45.736888 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:45.736847    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:46.091966 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.091929    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"]
Oct 31 03:57:46.123838 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:46.123804    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1751906b_97cd_4969_9817_4e5b9d0715bf.slice/crio-3ff5b22b3c148a935b16f7df6b575ea135bc58db2bf6179f714661e7efec0748 WatchSource:0}: Error finding container 3ff5b22b3c148a935b16f7df6b575ea135bc58db2bf6179f714661e7efec0748: Status 404 returned error can't find the container with id 3ff5b22b3c148a935b16f7df6b575ea135bc58db2bf6179f714661e7efec0748
Oct 31 03:57:46.139912 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.139882    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"]
Oct 31 03:57:46.141038 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:46.141015    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b24695_854c_413f_a5af_43558c164c55.slice/crio-563b312e00024f9b5df8beaed84fbd290335e055c6d9ecc6be577fbb06f9f6ca WatchSource:0}: Error finding container 563b312e00024f9b5df8beaed84fbd290335e055c6d9ecc6be577fbb06f9f6ca: Status 404 returned error can't find the container with id 563b312e00024f9b5df8beaed84fbd290335e055c6d9ecc6be577fbb06f9f6ca
Oct 31 03:57:46.937342 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.937299    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg" event={"ID":"32b24695-854c-413f-a5af-43558c164c55","Type":"ContainerStarted","Data":"da6bebf4c871f92e6cfb628cfb8c5b4b400ec365fa176cafbbf9af48f3a39c03"}
Oct 31 03:57:46.937790 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.937360    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg" event={"ID":"32b24695-854c-413f-a5af-43558c164c55","Type":"ContainerStarted","Data":"563b312e00024f9b5df8beaed84fbd290335e055c6d9ecc6be577fbb06f9f6ca"}
Oct 31 03:57:46.937790 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.937397    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:46.939020 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.938992    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws" event={"ID":"1751906b-97cd-4969-9817-4e5b9d0715bf","Type":"ContainerStarted","Data":"bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037"}
Oct 31 03:57:46.939158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.939036    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws" event={"ID":"1751906b-97cd-4969-9817-4e5b9d0715bf","Type":"ContainerStarted","Data":"3ff5b22b3c148a935b16f7df6b575ea135bc58db2bf6179f714661e7efec0748"}
Oct 31 03:57:46.939158 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:46.939064    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:47.061132 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:47.061076    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg" podStartSLOduration=2.061060521 podStartE2EDuration="2.061060521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:47.056917455 +0000 UTC m=+1716.890777808" watchObservedRunningTime="2026-10-31 03:57:47.061060521 +0000 UTC m=+1716.894920820"
Oct 31 03:57:47.197274 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:47.197136    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws" podStartSLOduration=3.197113158 podStartE2EDuration="3.197113158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:47.190191958 +0000 UTC m=+1717.024052264" watchObservedRunningTime="2026-10-31 03:57:47.197113158 +0000 UTC m=+1717.030973463"
Oct 31 03:57:48.093754 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:48.093722    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:50.504556 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:50.504472    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:57:50.504953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:50.504635    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:57:52.600133 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:52.600094    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:54.928526 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:54.928487    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 03:57:54.929031 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:54.928632    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 03:57:55.214557 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.214462    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"]
Oct 31 03:57:55.234667 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.234634    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.244377 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.244344    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"]
Oct 31 03:57:55.249932 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.249905    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.283312 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.283272    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"]
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326282    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326344    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl5cb\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326381    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326496    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326526    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326574    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326611    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326650    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326675    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326737    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326762    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326826    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.326916    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.327081    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jt7s\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.327164    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.327230    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.327277    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.329285 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.327343    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.381769 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.381406    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"]
Oct 31 03:57:55.428876 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.428836    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.429068 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.428912    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.429068 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.428953    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.429068 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.428980    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.429068 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429049    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.429357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429081    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl5cb\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.429357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429137    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.429357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429277    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.429357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429293    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.429357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429310    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.429522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429370    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.429522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.429436    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430066    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430131    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430172    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430261    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430317    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430388    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430417    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430482    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430498    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430557    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430591    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430676    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430718    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430757    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jt7s\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.430833    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.432142 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.431559    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.433494 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.433468    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.433600 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.433469    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.433941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.433900    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.435268 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.435197    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.492693 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.492657    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.492930 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.492904    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.493367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.493339    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jt7s\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s\") pod \"maas-default-gateway-openshift-default-7854949c9-7wm5c\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.493502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.493426    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl5cb\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb\") pod \"maas-default-gateway-openshift-default-7854949c9-kl6kq\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:55.558224 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.558169    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:55.570265 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:55.570237    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:56.001189 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:56.001156    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda64dedea_79ce_4aab_b35d_01fb3d0a5f5f.slice/crio-b0741c8d06988325b27118e154f4be863c6f060d616282c470f7dbcce4fb3fbe WatchSource:0}: Error finding container b0741c8d06988325b27118e154f4be863c6f060d616282c470f7dbcce4fb3fbe: Status 404 returned error can't find the container with id b0741c8d06988325b27118e154f4be863c6f060d616282c470f7dbcce4fb3fbe
Oct 31 03:57:56.003712 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.003666    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:56.003832 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.003787    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:56.003897 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.003833    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:56.004023 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.004001    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"]
Oct 31 03:57:56.009891 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.009829    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"]
Oct 31 03:57:56.017724 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.017646    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" event={"ID":"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f","Type":"ContainerStarted","Data":"b0741c8d06988325b27118e154f4be863c6f060d616282c470f7dbcce4fb3fbe"}
Oct 31 03:57:56.032723 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:57:56.032691    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod144777ed_8620_4ddd_99a8_514d12f2510a.slice/crio-2564e7228f76b7d5a7ca24309797bfb776a190992ddf734af5c45838ff120050 WatchSource:0}: Error finding container 2564e7228f76b7d5a7ca24309797bfb776a190992ddf734af5c45838ff120050: Status 404 returned error can't find the container with id 2564e7228f76b7d5a7ca24309797bfb776a190992ddf734af5c45838ff120050
Oct 31 03:57:56.035700 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.035666    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:56.035799 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.035742    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:56.035799 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:56.035773    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:57:57.026938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.026902    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" event={"ID":"144777ed-8620-4ddd-99a8-514d12f2510a","Type":"ContainerStarted","Data":"98b7519e854771c0942b5522ed15de37d8cc6c098727253d47f7fba9c297c40f"}
Oct 31 03:57:57.027445 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.026959    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" event={"ID":"144777ed-8620-4ddd-99a8-514d12f2510a","Type":"ContainerStarted","Data":"2564e7228f76b7d5a7ca24309797bfb776a190992ddf734af5c45838ff120050"}
Oct 31 03:57:57.028583 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.028563    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" event={"ID":"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f","Type":"ContainerStarted","Data":"e01deaab1867b80d861aeb0e48a869d0a6cf9370e6a17ec3bfbfe77214555a68"}
Oct 31 03:57:57.110941 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.110913    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:57:57.179624 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.179539    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podStartSLOduration=2.179510953 podStartE2EDuration="2.179510953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:57.175664483 +0000 UTC m=+1727.009524790" watchObservedRunningTime="2026-10-31 03:57:57.179510953 +0000 UTC m=+1727.013371308"
Oct 31 03:57:57.353292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.349102    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podStartSLOduration=2.349079157 podStartE2EDuration="2.349079157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:57.320746758 +0000 UTC m=+1727.154607066" watchObservedRunningTime="2026-10-31 03:57:57.349079157 +0000 UTC m=+1727.182939462"
Oct 31 03:57:57.567078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.567035    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:57:57.567479 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.567455    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:57:57.567622 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.567599    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:57:57.581088 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.581021    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:57:57.581427 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.581399    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:57:57.581596 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.581571    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:57:57.996546 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.996514    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:57:57.996737 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:57.996646    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:57:58.580232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:58.564624    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:57:58.580232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:58.564684    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:57:58.580232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:58.573612    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:57:58.580232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:58.573669    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:57:59.571229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:59.567195    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:57:59.571229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:59.567419    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:57:59.590230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:59.586362    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:57:59.590230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:57:59.586433    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:58:00.563237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:00.563157    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:58:00.563422 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:00.563239    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:58:00.574237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:00.571589    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:58:00.574237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:00.571661    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:58:01.565231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:01.559637    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:58:01.565231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:01.559701    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:58:01.593437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:01.584429    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:58:01.593437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:01.584493    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:58:01.724072 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:01.724040    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:02.558610 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:02.558578    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:58:02.558790 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:02.558628    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:58:02.570996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:02.570957    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:58:02.571454 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:02.571005    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:58:03.558760 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:03.558718    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Oct 31 03:58:03.559053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:03.559016    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Oct 31 03:58:03.570811 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:03.570780    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Oct 31 03:58:03.570907 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:03.570838    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Oct 31 03:58:04.447419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.447359    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:04.835361 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.835333    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:58:04.836223 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.836185    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:58:04.837468 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.837444    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 03:58:04.875196 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.874328    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:58:04.875674 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.875558    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:58:04.877938 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:04.877901    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 03:58:06.210960 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:06.210932    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:06.419770 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:06.419732    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:58:10.251883 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.251850    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:10.477195 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.477163    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"]
Oct 31 03:58:10.483437 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.483415    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.537784 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.537702    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"]
Oct 31 03:58:10.584618 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.584585    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6c89303e-serving-cert\""
Oct 31 03:58:10.648421 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.648390    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.648595 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.648433    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svchk\" (UniqueName: \"kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.648646 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.648588    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.648690 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.648655    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.651264 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.651236    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.756755    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.756873    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.757550    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.757608    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svchk\" (UniqueName: \"kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:10.759579    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-serving-cert: secret "maas-api-e2e-derive-6c89303e-serving-cert" not found
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:10.759741    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls podName:306169e0-4feb-4fe2-9018-6cdfc3a93eef nodeName:}" failed. No retries permitted until 2026-10-31 03:58:11.259718611 +0000 UTC m=+1741.093578914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls") pod "maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" (UID: "306169e0-4feb-4fe2-9018-6cdfc3a93eef") : secret "maas-api-e2e-derive-6c89303e-serving-cert" not found
Oct 31 03:58:10.770254 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.760109    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.806459 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.806414    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.850590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.838747    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svchk\" (UniqueName: \"kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:10.851170 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.851130    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:11.001001 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:10.999449    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:11.301507 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:11.301466    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:11.304126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:11.304096    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\") pod \"maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:11.397979 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:11.397944    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:11.786289 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:58:11.786258    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306169e0_4feb_4fe2_9018_6cdfc3a93eef.slice/crio-d907b442c568e344405664d3bc336566c9be40f150583eb24c6e46d12eeb1ef9 WatchSource:0}: Error finding container d907b442c568e344405664d3bc336566c9be40f150583eb24c6e46d12eeb1ef9: Status 404 returned error can't find the container with id d907b442c568e344405664d3bc336566c9be40f150583eb24c6e46d12eeb1ef9
Oct 31 03:58:11.790383 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:11.790360    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"]
Oct 31 03:58:12.219475 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:12.219383    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" event={"ID":"306169e0-4feb-4fe2-9018-6cdfc3a93eef","Type":"ContainerStarted","Data":"d907b442c568e344405664d3bc336566c9be40f150583eb24c6e46d12eeb1ef9"}
Oct 31 03:58:13.228258 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:13.228219    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" event={"ID":"306169e0-4feb-4fe2-9018-6cdfc3a93eef","Type":"ContainerStarted","Data":"4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90"}
Oct 31 03:58:13.228660 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:13.228292    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:15.340016 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:15.339970    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:19.241179 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.241147    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 03:58:19.374497 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.374424    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" podStartSLOduration=8.451247019 podStartE2EDuration="9.37440187s" totalImagesPullingTime="923.154851ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:58:13.353506761 +0000 UTC m=+1743.187367066" watchObservedRunningTime="2026-10-31 03:58:19.37440187 +0000 UTC m=+1749.208262176"
Oct 31 03:58:19.766136 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.766104    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"]
Oct 31 03:58:19.772271 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.772248    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.829501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.829470    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-938507-serving-cert\""
Oct 31 03:58:19.829820 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.829795    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"]
Oct 31 03:58:19.845473 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.845441    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk55h\" (UniqueName: \"kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.845636 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.845586    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.845746 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.845724    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.845870 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.845854    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.942504 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.942457    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:19.948023 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.947996    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk55h\" (UniqueName: \"kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.948191 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.948109    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.948191 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.948184    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.948335 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.948312    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.948444 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:19.948426    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-serving-cert: secret "maas-api-e2e-shared-a-w5-938507-serving-cert" not found
Oct 31 03:58:19.948524 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:19.948494    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls podName:d99c7b2d-bfad-45c9-9691-6e888fab2082 nodeName:}" failed. No retries permitted until 2026-10-31 03:58:20.448473431 +0000 UTC m=+1750.282333716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls") pod "maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" (UID: "d99c7b2d-bfad-45c9-9691-6e888fab2082") : secret "maas-api-e2e-shared-a-w5-938507-serving-cert" not found
Oct 31 03:58:19.948886 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.948862    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:19.950703 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:19.950680    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:20.059709 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:20.059472    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk55h\" (UniqueName: \"kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:20.095590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:20.094603    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:20.494714 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:20.494680    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:20.497292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:20.497268    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:20.595653 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:20.595615    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:20.686463 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:20.686430    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:21.063815 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:58:21.063768    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd99c7b2d_bfad_45c9_9691_6e888fab2082.slice/crio-6afdd07f73bd23cca93ac9d7e77f796cfbd86fef6dcac79f440f630916204ac2 WatchSource:0}: Error finding container 6afdd07f73bd23cca93ac9d7e77f796cfbd86fef6dcac79f440f630916204ac2: Status 404 returned error can't find the container with id 6afdd07f73bd23cca93ac9d7e77f796cfbd86fef6dcac79f440f630916204ac2
Oct 31 03:58:21.067017 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:21.066992    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"]
Oct 31 03:58:21.347433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:21.347350    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" event={"ID":"d99c7b2d-bfad-45c9-9691-6e888fab2082","Type":"ContainerStarted","Data":"6afdd07f73bd23cca93ac9d7e77f796cfbd86fef6dcac79f440f630916204ac2"}
Oct 31 03:58:22.361673 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:22.361639    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" event={"ID":"d99c7b2d-bfad-45c9-9691-6e888fab2082","Type":"ContainerStarted","Data":"bc4b53ed8c923955cf0323ebefef85ce7d4ee8927e2017c00d9caffd45c26e3c"}
Oct 31 03:58:22.362061 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:22.361767    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:22.486101 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:22.486035    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" podStartSLOduration=2.59541143 podStartE2EDuration="3.486019386s" totalImagesPullingTime="890.607956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:58:22.482356181 +0000 UTC m=+1752.316216508" watchObservedRunningTime="2026-10-31 03:58:22.486019386 +0000 UTC m=+1752.319879691"
Oct 31 03:58:23.732400 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:23.732365    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:24.390046 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.390014    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"]
Oct 31 03:58:24.394906 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.394880    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.450073 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.450039    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"]
Oct 31 03:58:24.489578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.489538    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.489750 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.489713    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.489806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.489766    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjbqj\" (UniqueName: \"kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.490070 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.490045    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.497147 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.497120    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-eb5ce91f-serving-cert\""
Oct 31 03:58:24.588286 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.588249    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:24.591958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.591933    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.592078 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.592067    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.592141 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.592130    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.592179 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.592155    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjbqj\" (UniqueName: \"kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.592349 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:24.592322    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-serving-cert: secret "maas-api-e2e-ait-eb5ce91f-serving-cert" not found
Oct 31 03:58:24.592475 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:24.592387    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls podName:56eca937-06f6-4100-982b-62f3d992b7b4 nodeName:}" failed. No retries permitted until 2026-10-31 03:58:25.092369581 +0000 UTC m=+1754.926229881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls") pod "maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" (UID: "56eca937-06f6-4100-982b-62f3d992b7b4") : secret "maas-api-e2e-ait-eb5ce91f-serving-cert" not found
Oct 31 03:58:24.592686 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.592668    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.594580 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.594562    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.706276 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.705328    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjbqj\" (UniqueName: \"kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:24.736949 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.736908    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:24.981735 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:24.981659    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:25.121040 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:25.121006    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:25.123536 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:25.123507    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\") pod \"maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:25.310330 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:25.310292    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:25.685399 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:58:25.685357    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56eca937_06f6_4100_982b_62f3d992b7b4.slice/crio-bb0411ff0d1a5ceb3b9cd69f2d588b8ccc4baa911db57a5ec735bae4f8586621 WatchSource:0}: Error finding container bb0411ff0d1a5ceb3b9cd69f2d588b8ccc4baa911db57a5ec735bae4f8586621: Status 404 returned error can't find the container with id bb0411ff0d1a5ceb3b9cd69f2d588b8ccc4baa911db57a5ec735bae4f8586621
Oct 31 03:58:25.689372 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:25.689335    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"]
Oct 31 03:58:26.406356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:26.406321    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" event={"ID":"56eca937-06f6-4100-982b-62f3d992b7b4","Type":"ContainerStarted","Data":"bb0411ff0d1a5ceb3b9cd69f2d588b8ccc4baa911db57a5ec735bae4f8586621"}
Oct 31 03:58:27.455235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:27.455179    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" event={"ID":"56eca937-06f6-4100-982b-62f3d992b7b4","Type":"ContainerStarted","Data":"273d53adce803c7f8ac022e98e84d40df449b588e481e9635cf355af6542c3cb"}
Oct 31 03:58:27.460023 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:27.459993    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:27.589280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:27.589222    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" podStartSLOduration=2.67367565 podStartE2EDuration="3.589189584s" totalImagesPullingTime="915.513934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:58:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:58:27.586473779 +0000 UTC m=+1757.420334109" watchObservedRunningTime="2026-10-31 03:58:27.589189584 +0000 UTC m=+1757.423049888"
Oct 31 03:58:28.393996 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:28.393967    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 03:58:29.287956 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:29.287908    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:33.468319 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:33.468288    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 03:58:33.756862 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:33.756827    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:36.382791 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:58:36.382753    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:58:38.223357 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:38.223324    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:41.722311 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:41.722275    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:42.566502 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:42.566470    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:42.894252 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:42.894154    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:45.815918 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:45.815888    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:46.013808 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.013773    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:46.396353 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.396315    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"]
Oct 31 03:58:46.402352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.402328    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.456450 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.456415    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-976cea-openshift-default-dockercfg-6df8b\""
Oct 31 03:58:46.486262 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.486230    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:46.512501 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512463    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.512687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512585    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.512687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512638    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhpd4\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.512687 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512682    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.512896 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512809    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.512896 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512855    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.513010 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.512975    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.513071 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.513018    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.513219 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.513185    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.514766 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.514739    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"]
Oct 31 03:58:46.653755 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.630384    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.675374 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.675335    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.675816 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.675791    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.688230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.682385    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.698504 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.698463    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.698806 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.698782    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhpd4\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.705352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.705320    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.705594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.705577    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.710237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.705691    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.714233 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.632717    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.718301 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.711356    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.718500 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.679076    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.718629 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.683752    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.718770 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.679396    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.762853 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.760816    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.763766 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.763703    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.830234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.825438    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:46.865101 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.864978    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:46.886245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:46.882073    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhpd4\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4\") pod \"e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") " pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:47.039593 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:47.039562    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:47.486332 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:47.472167    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:47.827964 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:47.825289    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:47.984230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:47.984179    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:48.076194 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:58:48.076162    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749d4774_58db_40e5_bf02_cf4d37da86dc.slice/crio-88e9c63033f72dfe2fa708d01df85abda3de84c479dbdf3c2f047d67ac691f04 WatchSource:0}: Error finding container 88e9c63033f72dfe2fa708d01df85abda3de84c479dbdf3c2f047d67ac691f04: Status 404 returned error can't find the container with id 88e9c63033f72dfe2fa708d01df85abda3de84c479dbdf3c2f047d67ac691f04
Oct 31 03:58:48.079018 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:48.078991    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"]
Oct 31 03:58:48.079356 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:48.079327    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:58:48.079412 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:48.079388    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:58:48.079448 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:48.079416    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:58:48.921545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:48.921500    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" event={"ID":"749d4774-58db-40e5-bf02-cf4d37da86dc","Type":"ContainerStarted","Data":"0087d8aa6f291bfe325aae3f6ffff0c8cb85c62bb30e1cde9c7d8e711919175a"}
Oct 31 03:58:48.921820 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:48.921801    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" event={"ID":"749d4774-58db-40e5-bf02-cf4d37da86dc","Type":"ContainerStarted","Data":"88e9c63033f72dfe2fa708d01df85abda3de84c479dbdf3c2f047d67ac691f04"}
Oct 31 03:58:49.051381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:49.051351    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:49.057014 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:49.056987    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:49.102792 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:49.102739    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podStartSLOduration=3.102725169 podStartE2EDuration="3.102725169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:58:49.098438336 +0000 UTC m=+1778.932298676" watchObservedRunningTime="2026-10-31 03:58:49.102725169 +0000 UTC m=+1778.936585536"
Oct 31 03:58:49.928568 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:49.928536    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:49.929367 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:49.929345    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 03:58:51.085744 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:51.085704    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:51.895080 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:51.894992    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:52.250279 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:52.250244    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:52.502465 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:52.502352    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:56.858300 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:56.838119    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:57.029737 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:57.029694    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:58:57.186290 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:58:57.176362    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:00.210238 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:00.209063    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:00.850234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:00.849630    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:00.893229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:00.889316    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 03:59:00.917289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:00.902381    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 31 03:59:01.047234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:01.042338    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:59:02.005329 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:02.005228    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:02.209274 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:02.200939    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:02.375124 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:02.374874    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:02.599260 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:02.598684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:02.739679 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:02.739592    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:02.888187 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:02.888150    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:05.018172 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:05.018092    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:05.167127 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:05.166558    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:05.483281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:05.465443    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:05.890230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:05.886289    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:06.548040 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:06.548000    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:59:07.079702 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:07.079663    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:07.487524 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:07.487419    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:59:07.487695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:07.487513    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:59:11.881053 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:11.881021    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:12.229586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:12.022451    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:12.384338 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:12.368083    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:13.192592 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:13.192144    5906 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@7-28848-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe74473_2dc3_4d54_91b6_e93dbb6dee0b.slice/crio-conmon-708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe74473_2dc3_4d54_91b6_e93dbb6dee0b.slice/crio-708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae211c5c_03d3_4978_85ec_65f045d3d318.slice/crio-conmon-78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 03:59:13.713472 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.713436    5906 generic.go:356] "Generic (PLEG): container finished" podID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerID="708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54" exitCode=0
Oct 31 03:59:13.713672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.713507    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" event={"ID":"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b","Type":"ContainerDied","Data":"708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"}
Oct 31 03:59:13.714085 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.714058    5906 scope.go:122] "RemoveContainer" containerID="708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"
Oct 31 03:59:13.716077 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.715674    5906 generic.go:356] "Generic (PLEG): container finished" podID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerID="3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995" exitCode=0
Oct 31 03:59:13.716077 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.715798    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" event={"ID":"fc2f33f4-6678-4542-a9df-0a2a3781c5ab","Type":"ContainerDied","Data":"3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"}
Oct 31 03:59:13.729520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.716177    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.729520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.716309    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.729520 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.716355    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.730407 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.730311    5906 scope.go:122] "RemoveContainer" containerID="3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"
Oct 31 03:59:13.733547 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.733489    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.733865 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.733829    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.733983 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.733895    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.734255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.734173    5906 generic.go:356] "Generic (PLEG): container finished" podID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerID="78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac" exitCode=0
Oct 31 03:59:13.734997 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.734433    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" event={"ID":"ae211c5c-03d3-4978-85ec-65f045d3d318","Type":"ContainerDied","Data":"78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"}
Oct 31 03:59:13.736613 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.736558    5906 scope.go:122] "RemoveContainer" containerID="78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"
Oct 31 03:59:13.744914 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.744879    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.745033 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.744957    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:13.745033 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:13.744991    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:14.744590 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:14.744545    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" event={"ID":"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b","Type":"ContainerStarted","Data":"b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba"}
Oct 31 03:59:14.746901 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:14.746870    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" event={"ID":"fc2f33f4-6678-4542-a9df-0a2a3781c5ab","Type":"ContainerStarted","Data":"b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa"}
Oct 31 03:59:14.749046 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:14.749021    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" event={"ID":"ae211c5c-03d3-4978-85ec-65f045d3d318","Type":"ContainerStarted","Data":"6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996"}
Oct 31 03:59:15.143528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.140820    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:59:15.143528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.140996    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:15.143528 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.141041    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:15.333485 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.317534    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:59:15.333695 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.333630    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:15.333769 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.333689    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:15.631944 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.631887    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:59:15.633272 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.633241    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:15.633418 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:15.633301    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:16.140231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.137061    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:16.140231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.137134    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:16.341719 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.321140    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:16.354235 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.350300    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:16.632081 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.632035    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:16.632310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.632108    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:16.884295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:16.880383    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:17.145446 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.145348    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:17.146063 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.146023    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:17.313433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.313395    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:17.313780 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.313726    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:17.534415 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.534140    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:17.633243 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.632677    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:17.633243 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.632739    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:17.991287 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:17.981308    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:18.130115 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.130080    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:18.130333 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.130139    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:18.316767 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.316666    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:18.317409 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.317358    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:18.414381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.414345    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:59:18.419326 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.419292    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:59:18.633733 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.633615    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:18.634029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:18.633988    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:19.127230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.126447    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"]
Oct 31 03:59:19.221237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.175799    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:19.221237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.175863    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:19.272572 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.259491    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.277237 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.275319    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:19.296226 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.281346    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"]
Oct 31 03:59:19.333341 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.324370    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:19.333341 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.324451    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:19.355571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.355540    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w5-440092-openshift-default-dockercfg-twbrh\""
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.418975    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419051    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419118    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419194    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419522    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r78mt\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419614    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419723    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419793    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.422281 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.419876    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.493303 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.492122    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.515049    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.535672    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r78mt\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.535770    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.535876    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.535929    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.536009    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.536091    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.536145    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.536182    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.547723 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.536266    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.561775 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.561729    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.601890 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.591034    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.602325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.602258    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.602325 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.602196    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.603389 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.603363    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.661935 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.661896    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:19.662138 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.661975    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:19.757848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.742038    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.757848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.742592    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.757848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.749892    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.757848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.752644    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r78mt\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt\") pod \"e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") " pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:19.933577 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:19.933542    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:20.130505 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.130468    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:20.130713 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.130518    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:20.327352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.327315    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:20.327586 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.327560    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:20.632541 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.632449    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:20.632541 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.632524    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:20.921289 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.920366    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"]
Oct 31 03:59:20.930794 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:59:20.930751    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ae802c_e968_4325_9415_0ff3be0e670b.slice/crio-a7642c4847ce698e8dc1579d5ee41cccf38bb5f54cfd1b99a16ce7fed60883f2 WatchSource:0}: Error finding container a7642c4847ce698e8dc1579d5ee41cccf38bb5f54cfd1b99a16ce7fed60883f2: Status 404 returned error can't find the container with id a7642c4847ce698e8dc1579d5ee41cccf38bb5f54cfd1b99a16ce7fed60883f2
Oct 31 03:59:20.934569 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.934516    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:20.934685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.934607    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:20.934685 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:20.934656    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 03:59:21.166594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.151013    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:21.166594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.151077    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:21.289728 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.289685    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" event={"ID":"22ae802c-e968-4325-9415-0ff3be0e670b","Type":"ContainerStarted","Data":"a7642c4847ce698e8dc1579d5ee41cccf38bb5f54cfd1b99a16ce7fed60883f2"}
Oct 31 03:59:21.332253 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.325434    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:21.332253 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.325492    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:21.444898 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.442972    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:59:21.660245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.649881    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused" start-of-body=
Oct 31 03:59:21.660245 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:21.649941    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.93:15021/healthz/ready\": dial tcp 10.134.0.93:15021: connect: connection refused"
Oct 31 03:59:22.138231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.134801    5906 patch_prober.go:29] interesting pod/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Oct 31 03:59:22.138231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.134876    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Oct 31 03:59:22.138231 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.136638    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:22.332247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.331317    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 31 03:59:22.332247 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.331393    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 31 03:59:22.450707 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.450602    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" event={"ID":"22ae802c-e968-4325-9415-0ff3be0e670b","Type":"ContainerStarted","Data":"f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"}
Oct 31 03:59:22.570186 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.570124    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podStartSLOduration=4.570109285 podStartE2EDuration="4.570109285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:59:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:59:22.566352295 +0000 UTC m=+1812.400212601" watchObservedRunningTime="2026-10-31 03:59:22.570109285 +0000 UTC m=+1812.403969621"
Oct 31 03:59:22.934505 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.934463    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:22.940955 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:22.940925    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:23.051307 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.051273    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:59:23.061453 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.061419    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 03:59:23.136833 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.136801    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:59:23.138019 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.137982    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 03:59:23.335347 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.335314    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:59:23.336280 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.336250    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"]
Oct 31 03:59:23.337042 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.337021    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 03:59:23.428148 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.428081    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"]
Oct 31 03:59:23.452803 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.452770    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:23.453652 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.453630    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 03:59:23.662051 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.661973    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:23.803416 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:23.803379    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:24.068559 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:24.068521    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:25.541643 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:25.541596    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:25.705752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:25.705722    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:25.909945 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:25.909861    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:27.012617 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:27.012583    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:28.336974 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.336912    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb" podUID="cf6138a6-771e-4e1c-a54a-af370f42e346" containerName="payload-processing" containerID="cri-o://c0ae1f548ea2cf3053d91c823db78e06ef3a465a727607763720fbed10d134db" gracePeriod=25
Oct 31 03:59:28.428766 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.428727    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm" podUID="bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" containerName="payload-pre-processing" containerID="cri-o://44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b" gracePeriod=25
Oct 31 03:59:28.493571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.493538    5906 generic.go:356] "Generic (PLEG): container finished" podID="cf6138a6-771e-4e1c-a54a-af370f42e346" containerID="c0ae1f548ea2cf3053d91c823db78e06ef3a465a727607763720fbed10d134db" exitCode=0
Oct 31 03:59:28.493729 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.493617    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb" event={"ID":"cf6138a6-771e-4e1c-a54a-af370f42e346","Type":"ContainerDied","Data":"c0ae1f548ea2cf3053d91c823db78e06ef3a465a727607763720fbed10d134db"}
Oct 31 03:59:28.680466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.680411    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:28.748573 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.748548    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:59:28.871958 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.871669    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume\") pod \"cf6138a6-771e-4e1c-a54a-af370f42e346\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") "
Oct 31 03:59:28.875112 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.873762    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:28.876156 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.875803    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume" pod "cf6138a6-771e-4e1c-a54a-af370f42e346" (UID: "cf6138a6-771e-4e1c-a54a-af370f42e346"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:28.903668 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.902428    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp\" (UniqueName: \"kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp\") pod \"cf6138a6-771e-4e1c-a54a-af370f42e346\" (UID: \"cf6138a6-771e-4e1c-a54a-af370f42e346\") "
Oct 31 03:59:28.905323 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.905292    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf6138a6-771e-4e1c-a54a-af370f42e346-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:28.908953 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:28.908894    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp" pod "cf6138a6-771e-4e1c-a54a-af370f42e346" (UID: "cf6138a6-771e-4e1c-a54a-af370f42e346"). InnerVolumeSpecName "kube-api-access-rt6jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:29.011264 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.009695    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt6jp\" (UniqueName: \"kubernetes.io/projected/cf6138a6-771e-4e1c-a54a-af370f42e346-kube-api-access-rt6jp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:29.051302 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.037371    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:59:29.105224 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.104579    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:29.112530 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.110883    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4\" (UniqueName: \"kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4\") pod \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") "
Oct 31 03:59:29.112530 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.110994    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume\") pod \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\" (UID: \"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6\") "
Oct 31 03:59:29.113232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.113169    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume" pod "bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" (UID: "bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:29.114835 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.114029    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:29.168602 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.126531    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4" pod "bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" (UID: "bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6"). InnerVolumeSpecName "kube-api-access-vmvm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:29.250073 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.244698    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmvm4\" (UniqueName: \"kubernetes.io/projected/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6-kube-api-access-vmvm4\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:29.594229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.593376    5906 generic.go:356] "Generic (PLEG): container finished" podID="bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" containerID="44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b" exitCode=0
Oct 31 03:59:29.594229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.593655    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm" event={"ID":"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6","Type":"ContainerDied","Data":"44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b"}
Oct 31 03:59:29.594229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.593717    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"
Oct 31 03:59:29.594229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.593729    5906 scope.go:122] "RemoveContainer" containerID="44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b"
Oct 31 03:59:29.594229 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.593716    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm" event={"ID":"bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6","Type":"ContainerDied","Data":"7d0ca9aa7df8d1277188177c6746471f951b2a217423f6071ab04534ad575e49"}
Oct 31 03:59:29.605157 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.601795    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb" event={"ID":"cf6138a6-771e-4e1c-a54a-af370f42e346","Type":"ContainerDied","Data":"aea4a99a51d25135900d8121681984a87a9c2f04946e457eb31f530fab68e2b6"}
Oct 31 03:59:29.605157 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.601919    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"
Oct 31 03:59:29.623641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.623616    5906 scope.go:122] "RemoveContainer" containerID="44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b"
Oct 31 03:59:29.624295 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:29.624270    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b\": container with ID starting with 44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b not found: ID does not exist" containerID="44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b"
Oct 31 03:59:29.624471 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.624430    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b"} err="failed to get container status \"44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b\": rpc error: code = NotFound desc = could not find container \"44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b\": container with ID starting with 44b4d9f18c38cbf4aa5d3557b8e72e4ec750db5edde87295641d089c957c804b not found: ID does not exist"
Oct 31 03:59:29.624571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.624560    5906 scope.go:122] "RemoveContainer" containerID="c0ae1f548ea2cf3053d91c823db78e06ef3a465a727607763720fbed10d134db"
Oct 31 03:59:29.663027 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.662989    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:29.768646 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.768611    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"]
Oct 31 03:59:29.772478 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.772454    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6c89303e-568f576679-4grxm"]
Oct 31 03:59:29.883752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.883661    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"]
Oct 31 03:59:29.887720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:29.887692    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6c89303e-6bc97cb8fc-8qgnb"]
Oct 31 03:59:30.422822 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:30.422785    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" path="/var/lib/kubelet/pods/bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6/volumes"
Oct 31 03:59:30.423927 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:30.423531    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf6138a6-771e-4e1c-a54a-af370f42e346" path="/var/lib/kubelet/pods/cf6138a6-771e-4e1c-a54a-af370f42e346/volumes"
Oct 31 03:59:32.196251 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:32.196190    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:34.646140 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:34.646107    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:34.810403 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:34.810365    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:36.386894 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:36.386852    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 03:59:36.454321 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:36.454289    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"]
Oct 31 03:59:36.563916 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:36.563880    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"]
Oct 31 03:59:36.787337 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:36.787303    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:36.924387 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:36.924351    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:37.080291 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:37.076916    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:40.674069 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:40.674027    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"]
Oct 31 03:59:40.674571 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:40.674325    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy" containerID="cri-o://47cfd9ed91d68864bb2ba1d8e687cdc3a249a165df75b8a02462692093b13d26" gracePeriod=30
Oct 31 03:59:41.455320 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:41.455263    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg" podUID="32b24695-854c-413f-a5af-43558c164c55" containerName="payload-processing" containerID="cri-o://da6bebf4c871f92e6cfb628cfb8c5b4b400ec365fa176cafbbf9af48f3a39c03" gracePeriod=25
Oct 31 03:59:41.564152 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:41.564113    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws" podUID="1751906b-97cd-4969-9817-4e5b9d0715bf" containerName="payload-pre-processing" containerID="cri-o://bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037" gracePeriod=25
Oct 31 03:59:41.604585 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:41.604554    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:41.832992 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:41.830546    5906 generic.go:356] "Generic (PLEG): container finished" podID="32b24695-854c-413f-a5af-43558c164c55" containerID="da6bebf4c871f92e6cfb628cfb8c5b4b400ec365fa176cafbbf9af48f3a39c03" exitCode=0
Oct 31 03:59:41.832992 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:41.830656    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg" event={"ID":"32b24695-854c-413f-a5af-43558c164c55","Type":"ContainerDied","Data":"da6bebf4c871f92e6cfb628cfb8c5b4b400ec365fa176cafbbf9af48f3a39c03"}
Oct 31 03:59:41.880599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:41.880570    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:59:42.011295 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.011263    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd\" (UniqueName: \"kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd\") pod \"32b24695-854c-413f-a5af-43558c164c55\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") "
Oct 31 03:59:42.011686 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.011664    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume\") pod \"32b24695-854c-413f-a5af-43558c164c55\" (UID: \"32b24695-854c-413f-a5af-43558c164c55\") "
Oct 31 03:59:42.012140 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.012111    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume" pod "32b24695-854c-413f-a5af-43558c164c55" (UID: "32b24695-854c-413f-a5af-43558c164c55"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:42.013838 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.012536    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32b24695-854c-413f-a5af-43558c164c55-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:42.013838 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.013753    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd" pod "32b24695-854c-413f-a5af-43558c164c55" (UID: "32b24695-854c-413f-a5af-43558c164c55"). InnerVolumeSpecName "kube-api-access-p5ljd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:42.046516 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.046488    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:59:42.114144 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.114056    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume\") pod \"1751906b-97cd-4969-9817-4e5b9d0715bf\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") "
Oct 31 03:59:42.114351 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.114334    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2\" (UniqueName: \"kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2\") pod \"1751906b-97cd-4969-9817-4e5b9d0715bf\" (UID: \"1751906b-97cd-4969-9817-4e5b9d0715bf\") "
Oct 31 03:59:42.114522 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.114493    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume" pod "1751906b-97cd-4969-9817-4e5b9d0715bf" (UID: "1751906b-97cd-4969-9817-4e5b9d0715bf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:42.114892 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.114879    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1751906b-97cd-4969-9817-4e5b9d0715bf-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:42.114957 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.114894    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5ljd\" (UniqueName: \"kubernetes.io/projected/32b24695-854c-413f-a5af-43558c164c55-kube-api-access-p5ljd\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:42.116506 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.116480    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2" pod "1751906b-97cd-4969-9817-4e5b9d0715bf" (UID: "1751906b-97cd-4969-9817-4e5b9d0715bf"). InnerVolumeSpecName "kube-api-access-jsgj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:42.216841 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.216797    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jsgj2\" (UniqueName: \"kubernetes.io/projected/1751906b-97cd-4969-9817-4e5b9d0715bf-kube-api-access-jsgj2\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:42.838824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.838790    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"
Oct 31 03:59:42.839345 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.838793    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg" event={"ID":"32b24695-854c-413f-a5af-43558c164c55","Type":"ContainerDied","Data":"563b312e00024f9b5df8beaed84fbd290335e055c6d9ecc6be577fbb06f9f6ca"}
Oct 31 03:59:42.839345 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.838947    5906 scope.go:122] "RemoveContainer" containerID="da6bebf4c871f92e6cfb628cfb8c5b4b400ec365fa176cafbbf9af48f3a39c03"
Oct 31 03:59:42.840530 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.840504    5906 generic.go:356] "Generic (PLEG): container finished" podID="1751906b-97cd-4969-9817-4e5b9d0715bf" containerID="bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037" exitCode=0
Oct 31 03:59:42.840651 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.840550    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"
Oct 31 03:59:42.840651 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.840563    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws" event={"ID":"1751906b-97cd-4969-9817-4e5b9d0715bf","Type":"ContainerDied","Data":"bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037"}
Oct 31 03:59:42.840651 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.840618    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws" event={"ID":"1751906b-97cd-4969-9817-4e5b9d0715bf","Type":"ContainerDied","Data":"3ff5b22b3c148a935b16f7df6b575ea135bc58db2bf6179f714661e7efec0748"}
Oct 31 03:59:42.850305 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.850096    5906 scope.go:122] "RemoveContainer" containerID="bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037"
Oct 31 03:59:42.859651 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.859630    5906 scope.go:122] "RemoveContainer" containerID="bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037"
Oct 31 03:59:42.859923 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:42.859901    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037\": container with ID starting with bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037 not found: ID does not exist" containerID="bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037"
Oct 31 03:59:42.859986 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:42.859933    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037"} err="failed to get container status \"bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037\": rpc error: code = NotFound desc = could not find container \"bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037\": container with ID starting with bb8b2fada85c9e30105c14474e55184f42e8aa5a314f9e64531027fab4770037 not found: ID does not exist"
Oct 31 03:59:43.011570 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:43.011532    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"]
Oct 31 03:59:43.063277 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:43.063240    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-eb5ce91f-76ffc87cff-7cgtg"]
Oct 31 03:59:43.131380 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:43.131280    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"]
Oct 31 03:59:43.135405 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:43.135376    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-eb5ce91f-7c6dfbd959-qmtws"]
Oct 31 03:59:44.358225 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:44.358174    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1751906b-97cd-4969-9817-4e5b9d0715bf" path="/var/lib/kubelet/pods/1751906b-97cd-4969-9817-4e5b9d0715bf/volumes"
Oct 31 03:59:44.361480 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:44.359652    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32b24695-854c-413f-a5af-43558c164c55" path="/var/lib/kubelet/pods/32b24695-854c-413f-a5af-43558c164c55/volumes"
Oct 31 03:59:45.661464 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:45.661429    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:45.872891 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:45.872855    5906 generic.go:356] "Generic (PLEG): container finished" podID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerID="47cfd9ed91d68864bb2ba1d8e687cdc3a249a165df75b8a02462692093b13d26" exitCode=0
Oct 31 03:59:45.873087 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:45.873015    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" event={"ID":"6f04960d-4eff-4aeb-adfa-3f547833d2ee","Type":"ContainerDied","Data":"47cfd9ed91d68864bb2ba1d8e687cdc3a249a165df75b8a02462692093b13d26"}
Oct 31 03:59:45.980328 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:45.980295    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:59:46.086965 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.086922    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087160 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.086975    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087160 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087038    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087167    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087199    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087310 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087293    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.087483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087361    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087483 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087407    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087601 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087506    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087601 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087590    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo\") pod \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\" (UID: \"6f04960d-4eff-4aeb-adfa-3f547833d2ee\") "
Oct 31 03:59:46.087691 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087612    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.087691 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087650    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:46.087821 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087803    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.087922 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.087898    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.088456 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.088433    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.088456 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.088454    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.088649 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.088464    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.088649 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.088472    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.088649 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.088480    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.089704 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.089679    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "kube-api-access-nlk88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:46.089824 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.089766    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.089946 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.089924    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:46.090327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.090301    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:46.090479 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.090458    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo" pod "6f04960d-4eff-4aeb-adfa-3f547833d2ee" (UID: "6f04960d-4eff-4aeb-adfa-3f547833d2ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:59:46.195049 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.194943    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.195049 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.194993    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.195049 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.195009    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlk88\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-kube-api-access-nlk88\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.195049 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.195026    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f04960d-4eff-4aeb-adfa-3f547833d2ee-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.882265 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.882200    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"
Oct 31 03:59:46.882814 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.882238    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp" event={"ID":"6f04960d-4eff-4aeb-adfa-3f547833d2ee","Type":"ContainerDied","Data":"af0c3a5ea9fbeadc425bff7f004ad7f6d51f20b998dbc422bed39c3882544ac6"}
Oct 31 03:59:46.882814 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:46.882362    5906 scope.go:122] "RemoveContainer" containerID="47cfd9ed91d68864bb2ba1d8e687cdc3a249a165df75b8a02462692093b13d26"
Oct 31 03:59:47.107029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:47.106985    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"]
Oct 31 03:59:47.159616 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:47.159516    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-mzrpp"]
Oct 31 03:59:48.354641 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:48.354598    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" path="/var/lib/kubelet/pods/6f04960d-4eff-4aeb-adfa-3f547833d2ee/volumes"
Oct 31 03:59:49.814765 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:49.814728    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"]
Oct 31 03:59:49.815256 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:49.814934    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy" containerID="cri-o://88a26dc39cd18dc9c0b2b0c0bd748831b723d665f284a38c1fd78a96d0aa34cf" gracePeriod=30
Oct 31 03:59:49.868580 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:49.868537    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"]
Oct 31 03:59:49.868853 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:49.868822    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy" containerID="cri-o://d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e" gracePeriod=30
Oct 31 03:59:50.291341 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.291306    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"]
Oct 31 03:59:50.292173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292150    5906 cpu_manager.go:414] "containerMap: removing container" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy"
Oct 31 03:59:50.292173 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292174    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292200    5906 cpu_manager.go:414] "containerMap: removing container" podUID="cf6138a6-771e-4e1c-a54a-af370f42e346" containerName="payload-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292224    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf6138a6-771e-4e1c-a54a-af370f42e346" containerName="payload-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292240    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1751906b-97cd-4969-9817-4e5b9d0715bf" containerName="payload-pre-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292247    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1751906b-97cd-4969-9817-4e5b9d0715bf" containerName="payload-pre-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292286    5906 cpu_manager.go:414] "containerMap: removing container" podUID="bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" containerName="payload-pre-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292293    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" containerName="payload-pre-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292309    5906 cpu_manager.go:414] "containerMap: removing container" podUID="32b24695-854c-413f-a5af-43558c164c55" containerName="payload-processing"
Oct 31 03:59:50.292381 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292316    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32b24695-854c-413f-a5af-43558c164c55" containerName="payload-processing"
Oct 31 03:59:50.292761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292531    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="6f04960d-4eff-4aeb-adfa-3f547833d2ee" containerName="istio-proxy"
Oct 31 03:59:50.292761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292550    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf6138a6-771e-4e1c-a54a-af370f42e346" containerName="payload-processing"
Oct 31 03:59:50.292761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292560    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1751906b-97cd-4969-9817-4e5b9d0715bf" containerName="payload-pre-processing"
Oct 31 03:59:50.292761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292576    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd2b0cc8-7f41-4bf1-8fc0-a4bac900d7c6" containerName="payload-pre-processing"
Oct 31 03:59:50.292761 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.292586    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="32b24695-854c-413f-a5af-43558c164c55" containerName="payload-processing"
Oct 31 03:59:50.298055 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.298033    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.352090 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.352057    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-22e20532-serving-cert\""
Oct 31 03:59:50.357834 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.357800    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"]
Oct 31 03:59:50.376093 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.376064    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.376292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.376120    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzvmc\" (UniqueName: \"kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.376292 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.376287    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.376433 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.376421    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.458969 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.458935    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:50.477784 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.477734    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.478004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.477932    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.478125 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.478103    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzvmc\" (UniqueName: \"kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.478385 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.478357    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.478584 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.478563    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.478664 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:50.478565    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-serving-cert: secret "maas-api-e2e-mt-22e20532-serving-cert" not found
Oct 31 03:59:50.478664 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:50.478660    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls podName:e6e690ce-54a5-4d88-8a7e-bc648fe9b483 nodeName:}" failed. No retries permitted until 2026-10-31 03:59:50.978640877 +0000 UTC m=+1840.812501187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls") pod "maas-api-e2e-mt-22e20532-86f689d595-7pjvq" (UID: "e6e690ce-54a5-4d88-8a7e-bc648fe9b483") : secret "maas-api-e2e-mt-22e20532-serving-cert" not found
Oct 31 03:59:50.480423 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.480401    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.540669 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.540636    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzvmc\" (UniqueName: \"kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.615110 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.615016    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:50.753966 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.753916    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:50.989516 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.989433    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:50.992046 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:50.992021    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\") pod \"maas-api-e2e-mt-22e20532-86f689d595-7pjvq\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:51.213843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:51.213804    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:51.592964 ip-10-0-129-108 kubenswrapper[5906]: W1031 03:59:51.592929    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e690ce_54a5_4d88_8a7e_bc648fe9b483.slice/crio-ee7c608b6724248a3aeb4538fc7c165d919ad9fe2658a7e4182b749f5971f972 WatchSource:0}: Error finding container ee7c608b6724248a3aeb4538fc7c165d919ad9fe2658a7e4182b749f5971f972: Status 404 returned error can't find the container with id ee7c608b6724248a3aeb4538fc7c165d919ad9fe2658a7e4182b749f5971f972
Oct 31 03:59:51.596826 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:51.596800    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"]
Oct 31 03:59:51.922094 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:51.922007    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq" event={"ID":"e6e690ce-54a5-4d88-8a7e-bc648fe9b483","Type":"ContainerStarted","Data":"ee7c608b6724248a3aeb4538fc7c165d919ad9fe2658a7e4182b749f5971f972"}
Oct 31 03:59:52.931375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:52.931275    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq" event={"ID":"e6e690ce-54a5-4d88-8a7e-bc648fe9b483","Type":"ContainerStarted","Data":"31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105"}
Oct 31 03:59:52.931848 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:52.931407    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:53.053375 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:53.053319    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq" podStartSLOduration=2.124807564 podStartE2EDuration="3.053304227s" totalImagesPullingTime="928.496663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:59:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:59:53.051853347 +0000 UTC m=+1842.885713652" watchObservedRunningTime="2026-10-31 03:59:53.053304227 +0000 UTC m=+1842.887164532"
Oct 31 03:59:54.948843 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:54.948795    5906 generic.go:356] "Generic (PLEG): container finished" podID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerID="88a26dc39cd18dc9c0b2b0c0bd748831b723d665f284a38c1fd78a96d0aa34cf" exitCode=0
Oct 31 03:59:54.949336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:54.948872    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" event={"ID":"025dfe4c-2482-43cf-bdac-a8e6f273a556","Type":"ContainerDied","Data":"88a26dc39cd18dc9c0b2b0c0bd748831b723d665f284a38c1fd78a96d0aa34cf"}
Oct 31 03:59:55.131464 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.130343    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:55.309466 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.307879    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:59:55.334230 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.312778    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:59:55.386684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386590    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.386684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386644    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.386937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386717    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.386937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386740    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.386937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386763    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.386937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386788    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.386937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386815    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.386937 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.386872    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387050    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387093    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387117    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387175    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387262    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387419    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387430    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387456    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387480    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387505    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387537    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387566    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy\") pod \"1a7799e3-3549-488c-becd-5d4d29d98874\" (UID: \"1a7799e3-3549-488c-becd-5d4d29d98874\") "
Oct 31 03:59:55.387684 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387625    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data\") pod \"025dfe4c-2482-43cf-bdac-a8e6f273a556\" (UID: \"025dfe4c-2482-43cf-bdac-a8e6f273a556\") "
Oct 31 03:59:55.388388 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.387806    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.388388 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388022    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.388388 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388229    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.388599 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388573    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.388661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388605    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a7799e3-3549-488c-becd-5d4d29d98874-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.388661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388620    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.388661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388633    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/025dfe4c-2482-43cf-bdac-a8e6f273a556-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.388661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388646    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.388661 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388660    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.388924 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.388900    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.389118 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.389093    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.389339 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.389318    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.389830 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.389789    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.390232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.389948    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:55.390232 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.389997    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:59:55.390400 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.390280    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "kube-api-access-r8nkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:55.391029 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.390998    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "kube-api-access-hh4ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:55.391891 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.391867    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.391997 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.391973    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:59:55.392336 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.392311    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy" pod "025dfe4c-2482-43cf-bdac-a8e6f273a556" (UID: "025dfe4c-2482-43cf-bdac-a8e6f273a556"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:55.392752 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.392725    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token" pod "1a7799e3-3549-488c-becd-5d4d29d98874" (UID: "1a7799e3-3549-488c-becd-5d4d29d98874"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:55.490308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490275    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8nkj\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-kube-api-access-r8nkj\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490304    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490308 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490314    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490322    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490332    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490340    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490348    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490356    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/025dfe4c-2482-43cf-bdac-a8e6f273a556-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490364    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a7799e3-3549-488c-becd-5d4d29d98874-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490372    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a7799e3-3549-488c-becd-5d4d29d98874-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490381    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.490642 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.490389    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh4ls\" (UniqueName: \"kubernetes.io/projected/1a7799e3-3549-488c-becd-5d4d29d98874-kube-api-access-hh4ls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:55.959706 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.959675    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"
Oct 31 03:59:55.960191 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.959682    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z" event={"ID":"025dfe4c-2482-43cf-bdac-a8e6f273a556","Type":"ContainerDied","Data":"ae54173c9305924f76243e8f6929c99fd4a04aae106aa96be093e6cf16846bb3"}
Oct 31 03:59:55.960191 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.959830    5906 scope.go:122] "RemoveContainer" containerID="88a26dc39cd18dc9c0b2b0c0bd748831b723d665f284a38c1fd78a96d0aa34cf"
Oct 31 03:59:55.961469 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.961437    5906 generic.go:356] "Generic (PLEG): container finished" podID="1a7799e3-3549-488c-becd-5d4d29d98874" containerID="d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e" exitCode=0
Oct 31 03:59:55.961594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.961485    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"
Oct 31 03:59:55.961594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.961508    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" event={"ID":"1a7799e3-3549-488c-becd-5d4d29d98874","Type":"ContainerDied","Data":"d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e"}
Oct 31 03:59:55.961594 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.961580    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw" event={"ID":"1a7799e3-3549-488c-becd-5d4d29d98874","Type":"ContainerDied","Data":"3542c42d02e0ac6e1f4b4db94700be182ab207652f0337ddadb194cc02244aaa"}
Oct 31 03:59:55.972434 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.972378    5906 scope.go:122] "RemoveContainer" containerID="d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e"
Oct 31 03:59:55.983664 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.983494    5906 scope.go:122] "RemoveContainer" containerID="d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e"
Oct 31 03:59:55.983828 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:55.983810    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e\": container with ID starting with d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e not found: ID does not exist" containerID="d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e"
Oct 31 03:59:55.983877 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:55.983839    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e"} err="failed to get container status \"d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e\": rpc error: code = NotFound desc = could not find container \"d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e\": container with ID starting with d968ae6dc11a4179a2262c1703eeec2468318f89fc4c6e64a09eba42c9e6bc0e not found: ID does not exist"
Oct 31 03:59:56.132672 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:56.132591    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"]
Oct 31 03:59:56.145982 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:56.145951    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-zm9bw"]
Oct 31 03:59:56.251720 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:56.251680    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"]
Oct 31 03:59:56.257620 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:56.257588    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-sxk2z"]
Oct 31 03:59:56.355871 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:56.355836    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" path="/var/lib/kubelet/pods/025dfe4c-2482-43cf-bdac-a8e6f273a556/volumes"
Oct 31 03:59:56.356536 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:56.356518    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" path="/var/lib/kubelet/pods/1a7799e3-3549-488c-becd-5d4d29d98874/volumes"
Oct 31 03:59:58.944352 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:58.944318    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 03:59:59.369511 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.369473    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"]
Oct 31 03:59:59.370255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.370232    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy"
Oct 31 03:59:59.370255 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.370248    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy"
Oct 31 03:59:59.370343 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.370276    5906 cpu_manager.go:414] "containerMap: removing container" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy"
Oct 31 03:59:59.370343 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.370281    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy"
Oct 31 03:59:59.370419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.370405    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a7799e3-3549-488c-becd-5d4d29d98874" containerName="istio-proxy"
Oct 31 03:59:59.370419 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.370416    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="025dfe4c-2482-43cf-bdac-a8e6f273a556" containerName="istio-proxy"
Oct 31 03:59:59.375820 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.375799    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.429271 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.429235    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"]
Oct 31 03:59:59.439004 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.438975    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-b4a3a5-serving-cert\""
Oct 31 03:59:59.457578 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.457544    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.457813 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.457789    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.458126 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.458096    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcs8w\" (UniqueName: \"kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.458545 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.458514    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.549234 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.549176    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:59.560940 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.560898    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.561112 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.561052    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.561170 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.561157    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.561278 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:59.561053    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-serving-cert: secret "maas-api-e2e-shared-a-w1-b4a3a5-serving-cert" not found
Oct 31 03:59:59.561278 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.561260    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcs8w\" (UniqueName: \"kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.561399 ip-10-0-129-108 kubenswrapper[5906]: E1031 03:59:59.561309    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls podName:852a17b4-91c5-4976-8be8-677229750cf4 nodeName:}" failed. No retries permitted until 2026-10-31 04:00:00.061283557 +0000 UTC m=+1849.895143860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls") pod "maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" (UID: "852a17b4-91c5-4976-8be8-677229750cf4") : secret "maas-api-e2e-shared-a-w1-b4a3a5-serving-cert" not found
Oct 31 03:59:59.561902 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.561875    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.563542 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.563523    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.706871 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.706777    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcs8w\" (UniqueName: \"kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 03:59:59.717327 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.717254    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 03:59:59.870222 ip-10-0-129-108 kubenswrapper[5906]: I1031 03:59:59.870124    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:00.112198 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.112163    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:00:00.114782 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.114753    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:00:00.212047 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.212011    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"]
Oct 31 04:00:00.216801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.216777    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.278328 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.278293    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"]
Oct 31 04:00:00.292321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.292289    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:00:00.316852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.316813    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.317420 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.317111    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpzvz\" (UniqueName: \"kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.325081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.325050    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:00:00.419713 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.419666    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpzvz\" (UniqueName: \"kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.419931 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.419909    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.420872 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.420845    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.484213 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.484168    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpzvz\" (UniqueName: \"kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.538153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.538117    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:00.708185 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:00.708154    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852a17b4_91c5_4976_8be8_677229750cf4.slice/crio-ea08ee773c594f0ecc35a11f8ddb150791783a7006edbe1584f5e53908d24a01 WatchSource:0}: Error finding container ea08ee773c594f0ecc35a11f8ddb150791783a7006edbe1584f5e53908d24a01: Status 404 returned error can't find the container with id ea08ee773c594f0ecc35a11f8ddb150791783a7006edbe1584f5e53908d24a01
Oct 31 04:00:00.712072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.712045    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"]
Oct 31 04:00:00.916530 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:00.916497    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40e0f92_ae7c_465e_b236_f7870b4dc4ba.slice/crio-835a535c06d086ae4480bc353e21333e8557badbe9dc4d6f2b661646ae8ac19d WatchSource:0}: Error finding container 835a535c06d086ae4480bc353e21333e8557badbe9dc4d6f2b661646ae8ac19d: Status 404 returned error can't find the container with id 835a535c06d086ae4480bc353e21333e8557badbe9dc4d6f2b661646ae8ac19d
Oct 31 04:00:00.920904 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:00.920875    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"]
Oct 31 04:00:01.002105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:01.002068    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" event={"ID":"852a17b4-91c5-4976-8be8-677229750cf4","Type":"ContainerStarted","Data":"ea08ee773c594f0ecc35a11f8ddb150791783a7006edbe1584f5e53908d24a01"}
Oct 31 04:00:01.003565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:01.003526    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" event={"ID":"b40e0f92-ae7c-465e-b236-f7870b4dc4ba","Type":"ContainerStarted","Data":"835a535c06d086ae4480bc353e21333e8557badbe9dc4d6f2b661646ae8ac19d"}
Oct 31 04:00:02.012859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:02.012820    5906 generic.go:356] "Generic (PLEG): container finished" podID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerID="f4b3a9820cbeda1a8de28f3d3c2d8a6e1fdb90b0007b010dd2427074309ee51c" exitCode=7
Oct 31 04:00:02.013367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:02.012904    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" event={"ID":"b40e0f92-ae7c-465e-b236-f7870b4dc4ba","Type":"ContainerDied","Data":"f4b3a9820cbeda1a8de28f3d3c2d8a6e1fdb90b0007b010dd2427074309ee51c"}
Oct 31 04:00:02.013367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:02.013244    5906 scope.go:122] "RemoveContainer" containerID="f4b3a9820cbeda1a8de28f3d3c2d8a6e1fdb90b0007b010dd2427074309ee51c"
Oct 31 04:00:03.022305 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.022269    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" event={"ID":"852a17b4-91c5-4976-8be8-677229750cf4","Type":"ContainerStarted","Data":"7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12"}
Oct 31 04:00:03.022793 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.022394    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:00:03.024472 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.024447    5906 generic.go:356] "Generic (PLEG): container finished" podID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerID="343b96ead5d8d7b2e0128627c39dec0e7a0ae822d7c14b31a45ba8f14c9ddb31" exitCode=7
Oct 31 04:00:03.024614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.024511    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" event={"ID":"b40e0f92-ae7c-465e-b236-f7870b4dc4ba","Type":"ContainerDied","Data":"343b96ead5d8d7b2e0128627c39dec0e7a0ae822d7c14b31a45ba8f14c9ddb31"}
Oct 31 04:00:03.024614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.024551    5906 scope.go:122] "RemoveContainer" containerID="f4b3a9820cbeda1a8de28f3d3c2d8a6e1fdb90b0007b010dd2427074309ee51c"
Oct 31 04:00:03.024885 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.024870    5906 scope.go:122] "RemoveContainer" containerID="343b96ead5d8d7b2e0128627c39dec0e7a0ae822d7c14b31a45ba8f14c9ddb31"
Oct 31 04:00:03.025308 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:03.025281    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47_odh-ai-gateway-infra(b40e0f92-ae7c-465e-b236-f7870b4dc4ba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba"
Oct 31 04:00:03.147241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:03.147168    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" podStartSLOduration=2.683544536 podStartE2EDuration="4.147149545s" totalImagesPullingTime="1.463605009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:03.143488772 +0000 UTC m=+1852.977349078" watchObservedRunningTime="2026-10-31 04:00:03.147149545 +0000 UTC m=+1852.981009851"
Oct 31 04:00:04.033819 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.033791    5906 scope.go:122] "RemoveContainer" containerID="343b96ead5d8d7b2e0128627c39dec0e7a0ae822d7c14b31a45ba8f14c9ddb31"
Oct 31 04:00:04.034244 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:04.034133    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47_odh-ai-gateway-infra(b40e0f92-ae7c-465e-b236-f7870b4dc4ba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba"
Oct 31 04:00:04.091748 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.091712    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"]
Oct 31 04:00:04.097662 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.097645    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.152045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.152011    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-976cea-serving-cert\""
Oct 31 04:00:04.155754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.155713    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"]
Oct 31 04:00:04.223776 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.223732    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.223964 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.223807    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djmkf\" (UniqueName: \"kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.224041 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.224005    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.224167 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.224148    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.273899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.273857    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:04.343285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.338330    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.381290 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:04.374931    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-serving-cert: secret "maas-api-e2e-shared-a-w3-976cea-serving-cert" not found
Oct 31 04:00:04.394848 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.390274    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djmkf\" (UniqueName: \"kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.394848 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.390541    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.394848 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.390767    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.398420 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:04.397730    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls podName:295a93f5-e299-4961-876d-d41f4528f5df nodeName:}" failed. No retries permitted until 2026-10-31 04:00:04.897702348 +0000 UTC m=+1854.731562638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls") pod "maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" (UID: "295a93f5-e299-4961-876d-d41f4528f5df") : secret "maas-api-e2e-shared-a-w3-976cea-serving-cert" not found
Oct 31 04:00:04.400884 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.398603    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.415228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.411993    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.440232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.439769    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:04.458854 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.458052    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djmkf\" (UniqueName: \"kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.603316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.594841    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:04.923589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.923488    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:04.926077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:04.926049    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:05.012303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:05.012263    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:05.399277 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:05.399227    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295a93f5_e299_4961_876d_d41f4528f5df.slice/crio-5c3d068fd5556a36992e5f47834f8921e581858c4cd81fa1945e4b033d9d6048 WatchSource:0}: Error finding container 5c3d068fd5556a36992e5f47834f8921e581858c4cd81fa1945e4b033d9d6048: Status 404 returned error can't find the container with id 5c3d068fd5556a36992e5f47834f8921e581858c4cd81fa1945e4b033d9d6048
Oct 31 04:00:05.403188 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:05.403161    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"]
Oct 31 04:00:06.052107 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:06.052069    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" event={"ID":"295a93f5-e299-4961-876d-d41f4528f5df","Type":"ContainerStarted","Data":"5c3d068fd5556a36992e5f47834f8921e581858c4cd81fa1945e4b033d9d6048"}
Oct 31 04:00:06.388532 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:06.388495    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:00:07.103227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:07.102563    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" event={"ID":"295a93f5-e299-4961-876d-d41f4528f5df","Type":"ContainerStarted","Data":"b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72"}
Oct 31 04:00:07.104006 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:07.103929    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:07.228973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:07.228902    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" podStartSLOduration=2.267529899 podStartE2EDuration="3.228883653s" totalImagesPullingTime="961.353754ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:07.225137043 +0000 UTC m=+1857.058997368" watchObservedRunningTime="2026-10-31 04:00:07.228883653 +0000 UTC m=+1857.062743960"
Oct 31 04:00:08.327758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:08.327718    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"]
Oct 31 04:00:08.328185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:08.327919    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" podUID="306169e0-4feb-4fe2-9018-6cdfc3a93eef" containerName="maas-api" containerID="cri-o://4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90" gracePeriod=30
Oct 31 04:00:08.417007 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:08.416966    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:08.992227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:08.988514    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 04:00:09.040039 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.039958    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:00:09.096447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.096411    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk\" (UniqueName: \"kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk\") pod \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") "
Oct 31 04:00:09.096635 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.096590    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\") pod \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") "
Oct 31 04:00:09.096695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.096684    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs\") pod \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") "
Oct 31 04:00:09.096746 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.096718    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca\") pod \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\" (UID: \"306169e0-4feb-4fe2-9018-6cdfc3a93eef\") "
Oct 31 04:00:09.097266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.097227    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca" pod "306169e0-4feb-4fe2-9018-6cdfc3a93eef" (UID: "306169e0-4feb-4fe2-9018-6cdfc3a93eef"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:09.097597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.097572    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/306169e0-4feb-4fe2-9018-6cdfc3a93eef-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:09.098828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.098801    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs" pod "306169e0-4feb-4fe2-9018-6cdfc3a93eef" (UID: "306169e0-4feb-4fe2-9018-6cdfc3a93eef"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:00:09.098828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.098815    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls" pod "306169e0-4feb-4fe2-9018-6cdfc3a93eef" (UID: "306169e0-4feb-4fe2-9018-6cdfc3a93eef"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:00:09.098954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.098896    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk" pod "306169e0-4feb-4fe2-9018-6cdfc3a93eef" (UID: "306169e0-4feb-4fe2-9018-6cdfc3a93eef"). InnerVolumeSpecName "kube-api-access-svchk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:09.126522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.123172    5906 generic.go:356] "Generic (PLEG): container finished" podID="306169e0-4feb-4fe2-9018-6cdfc3a93eef" containerID="4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90" exitCode=0
Oct 31 04:00:09.126522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.123287    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" event={"ID":"306169e0-4feb-4fe2-9018-6cdfc3a93eef","Type":"ContainerDied","Data":"4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90"}
Oct 31 04:00:09.126522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.123361    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5" event={"ID":"306169e0-4feb-4fe2-9018-6cdfc3a93eef","Type":"ContainerDied","Data":"d907b442c568e344405664d3bc336566c9be40f150583eb24c6e46d12eeb1ef9"}
Oct 31 04:00:09.126522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.123387    5906 scope.go:122] "RemoveContainer" containerID="4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90"
Oct 31 04:00:09.126522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.123301    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"
Oct 31 04:00:09.135560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.135536    5906 scope.go:122] "RemoveContainer" containerID="4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90"
Oct 31 04:00:09.135874 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:09.135854    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90\": container with ID starting with 4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90 not found: ID does not exist" containerID="4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90"
Oct 31 04:00:09.135952 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.135882    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90"} err="failed to get container status \"4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90\": rpc error: code = NotFound desc = could not find container \"4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90\": container with ID starting with 4efd876b3cab18d0428eeeda2c5771f43e2de602934bea8c58615135dfd5ce90 not found: ID does not exist"
Oct 31 04:00:09.199476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.199439    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:09.199476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.199470    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svchk\" (UniqueName: \"kubernetes.io/projected/306169e0-4feb-4fe2-9018-6cdfc3a93eef-kube-api-access-svchk\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:09.199476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.199481    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/306169e0-4feb-4fe2-9018-6cdfc3a93eef-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:09.499311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.491257    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"]
Oct 31 04:00:09.536438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.509531    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6c89303e-65b9c67b59-7gfk5"]
Oct 31 04:00:09.937873 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:09.937687    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:10.182055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:10.182007    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:10.338227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:10.331059    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:10.391682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:10.386705    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="306169e0-4feb-4fe2-9018-6cdfc3a93eef" path="/var/lib/kubelet/pods/306169e0-4feb-4fe2-9018-6cdfc3a93eef/volumes"
Oct 31 04:00:10.525255 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:10.525165    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:10.535842 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:10.535811    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"]
Oct 31 04:00:10.536364 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:10.536332    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy" containerID="cri-o://b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba" gracePeriod=30
Oct 31 04:00:14.129401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:14.129372    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:00:14.345938 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:14.345905    5906 scope.go:122] "RemoveContainer" containerID="343b96ead5d8d7b2e0128627c39dec0e7a0ae822d7c14b31a45ba8f14c9ddb31"
Oct 31 04:00:15.130249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.130184    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:15.171764 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.171714    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" event={"ID":"b40e0f92-ae7c-465e-b236-f7870b4dc4ba","Type":"ContainerStarted","Data":"05ee0dcac4fb5a9968dc186ebf5f6efcc8111de57fc105732cd207f6f6df36b8"}
Oct 31 04:00:15.822170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.822090    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 04:00:15.893024 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.892986    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893201 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893131    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893320 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893305    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893335    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893412 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893359    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893538 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893516    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893558    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893594    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893656    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo\") pod \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\" (UID: \"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b\") "
Oct 31 04:00:15.893768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893690    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:15.893971 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893924    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:15.893971 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.893938    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:15.894650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.894622    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.894757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.894654    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.894757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.894670    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.894757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.894736    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:15.894940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.894909    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:15.895711 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.895686    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:15.895814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.895721    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "kube-api-access-9vs8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:15.895814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.895753    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:15.896091 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.896063    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo" pod "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" (UID: "fbe74473-2dc3-4d54-91b6-e93dbb6dee0b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:15.996711 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.996674    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.996711 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.996712    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.996954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.996728    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.996954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.996742    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.996954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.996757    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:15.996954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:15.996766    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vs8f\" (UniqueName: \"kubernetes.io/projected/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b-kube-api-access-9vs8f\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:16.033718 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.033658    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" podStartSLOduration=16.033644675 podStartE2EDuration="16.033644675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:15.309642573 +0000 UTC m=+1865.143502881" watchObservedRunningTime="2026-10-31 04:00:16.033644675 +0000 UTC m=+1865.867505004"
Oct 31 04:00:16.180560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.180465    5906 generic.go:356] "Generic (PLEG): container finished" podID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerID="b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba" exitCode=0
Oct 31 04:00:16.180560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.180516    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"
Oct 31 04:00:16.180560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.180546    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" event={"ID":"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b","Type":"ContainerDied","Data":"b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba"}
Oct 31 04:00:16.181138 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.180601    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h" event={"ID":"fbe74473-2dc3-4d54-91b6-e93dbb6dee0b","Type":"ContainerDied","Data":"f49a7ec74b475fbcafb66dedd45c6802911eb9d272f5b0c1a6c2cd7121216a3d"}
Oct 31 04:00:16.181138 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.180618    5906 scope.go:122] "RemoveContainer" containerID="b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba"
Oct 31 04:00:16.182838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.182814    5906 generic.go:356] "Generic (PLEG): container finished" podID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerID="05ee0dcac4fb5a9968dc186ebf5f6efcc8111de57fc105732cd207f6f6df36b8" exitCode=60
Oct 31 04:00:16.182978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.182858    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" event={"ID":"b40e0f92-ae7c-465e-b236-f7870b4dc4ba","Type":"ContainerDied","Data":"05ee0dcac4fb5a9968dc186ebf5f6efcc8111de57fc105732cd207f6f6df36b8"}
Oct 31 04:00:16.183396 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.183376    5906 scope.go:122] "RemoveContainer" containerID="05ee0dcac4fb5a9968dc186ebf5f6efcc8111de57fc105732cd207f6f6df36b8"
Oct 31 04:00:16.183728 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:16.183710    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47_odh-ai-gateway-infra(b40e0f92-ae7c-465e-b236-f7870b4dc4ba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba"
Oct 31 04:00:16.192687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.192665    5906 scope.go:122] "RemoveContainer" containerID="708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"
Oct 31 04:00:16.202863 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.202836    5906 scope.go:122] "RemoveContainer" containerID="b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba"
Oct 31 04:00:16.203226 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:16.203179    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba\": container with ID starting with b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba not found: ID does not exist" containerID="b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba"
Oct 31 04:00:16.203328 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.203232    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba"} err="failed to get container status \"b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba\": rpc error: code = NotFound desc = could not find container \"b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba\": container with ID starting with b6e02a2799aa8575e7ebe801a76505f11b9621270f46dcbf7ca6e770d320fcba not found: ID does not exist"
Oct 31 04:00:16.203328 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.203260    5906 scope.go:122] "RemoveContainer" containerID="708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"
Oct 31 04:00:16.203537 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:16.203521    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54\": container with ID starting with 708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54 not found: ID does not exist" containerID="708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"
Oct 31 04:00:16.203580 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.203540    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54"} err="failed to get container status \"708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54\": rpc error: code = NotFound desc = could not find container \"708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54\": container with ID starting with 708abcd674917f57e9a032e228ec14d3cba439cfcd5980f84328edbb7b5f2e54 not found: ID does not exist"
Oct 31 04:00:16.203580 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.203554    5906 scope.go:122] "RemoveContainer" containerID="343b96ead5d8d7b2e0128627c39dec0e7a0ae822d7c14b31a45ba8f14c9ddb31"
Oct 31 04:00:16.304952 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.304914    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"]
Oct 31 04:00:16.357518 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.357488    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"]
Oct 31 04:00:16.357957 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:16.357936    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6c89303e-openshift-default-68b5b96db5-fgx2h"]
Oct 31 04:00:17.372030 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.372003    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:17.419913 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.419874    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz\" (UniqueName: \"kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz\") pod \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") "
Oct 31 04:00:17.420102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.419927    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca\") pod \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\" (UID: \"b40e0f92-ae7c-465e-b236-f7870b4dc4ba\") "
Oct 31 04:00:17.420397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.420366    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca" pod "b40e0f92-ae7c-465e-b236-f7870b4dc4ba" (UID: "b40e0f92-ae7c-465e-b236-f7870b4dc4ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:17.421049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.421029    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-maas-api-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:17.422375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.422350    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz" pod "b40e0f92-ae7c-465e-b236-f7870b4dc4ba" (UID: "b40e0f92-ae7c-465e-b236-f7870b4dc4ba"). InnerVolumeSpecName "kube-api-access-hpzvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:17.523281 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:17.523244    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpzvz\" (UniqueName: \"kubernetes.io/projected/b40e0f92-ae7c-465e-b236-f7870b4dc4ba-kube-api-access-hpzvz\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:18.210518 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:18.210480    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"
Oct 31 04:00:18.210702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:18.210508    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47" event={"ID":"b40e0f92-ae7c-465e-b236-f7870b4dc4ba","Type":"ContainerDied","Data":"835a535c06d086ae4480bc353e21333e8557badbe9dc4d6f2b661646ae8ac19d"}
Oct 31 04:00:18.210702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:18.210610    5906 scope.go:122] "RemoveContainer" containerID="05ee0dcac4fb5a9968dc186ebf5f6efcc8111de57fc105732cd207f6f6df36b8"
Oct 31 04:00:18.354001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:18.353966    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" path="/var/lib/kubelet/pods/fbe74473-2dc3-4d54-91b6-e93dbb6dee0b/volumes"
Oct 31 04:00:18.387256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:18.387219    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"]
Oct 31 04:00:18.439245 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:18.439195    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-b4a3a5-29890320-hzl47"]
Oct 31 04:00:19.042417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.042387    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"]
Oct 31 04:00:19.042632 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.042607    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" podUID="56eca937-06f6-4100-982b-62f3d992b7b4" containerName="maas-api" containerID="cri-o://273d53adce803c7f8ac022e98e84d40df449b588e481e9635cf355af6542c3cb" gracePeriod=30
Oct 31 04:00:19.126086 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.125975    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:19.222962 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.222927    5906 generic.go:356] "Generic (PLEG): container finished" podID="56eca937-06f6-4100-982b-62f3d992b7b4" containerID="273d53adce803c7f8ac022e98e84d40df449b588e481e9635cf355af6542c3cb" exitCode=0
Oct 31 04:00:19.223163 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.222992    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" event={"ID":"56eca937-06f6-4100-982b-62f3d992b7b4","Type":"ContainerDied","Data":"273d53adce803c7f8ac022e98e84d40df449b588e481e9635cf355af6542c3cb"}
Oct 31 04:00:19.522296 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.522244    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 04:00:19.660232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.655263    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj\" (UniqueName: \"kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj\") pod \"56eca937-06f6-4100-982b-62f3d992b7b4\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") "
Oct 31 04:00:19.660232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.655396    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\") pod \"56eca937-06f6-4100-982b-62f3d992b7b4\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") "
Oct 31 04:00:19.660232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.655479    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs\") pod \"56eca937-06f6-4100-982b-62f3d992b7b4\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") "
Oct 31 04:00:19.660232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.655707    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca\") pod \"56eca937-06f6-4100-982b-62f3d992b7b4\" (UID: \"56eca937-06f6-4100-982b-62f3d992b7b4\") "
Oct 31 04:00:19.684231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.680930    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca" pod "56eca937-06f6-4100-982b-62f3d992b7b4" (UID: "56eca937-06f6-4100-982b-62f3d992b7b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:19.692422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.692062    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs" pod "56eca937-06f6-4100-982b-62f3d992b7b4" (UID: "56eca937-06f6-4100-982b-62f3d992b7b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:00:19.721231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.717435    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj" pod "56eca937-06f6-4100-982b-62f3d992b7b4" (UID: "56eca937-06f6-4100-982b-62f3d992b7b4"). InnerVolumeSpecName "kube-api-access-tjbqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:19.739275 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.738421    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls" pod "56eca937-06f6-4100-982b-62f3d992b7b4" (UID: "56eca937-06f6-4100-982b-62f3d992b7b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:00:19.770231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.770108    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjbqj\" (UniqueName: \"kubernetes.io/projected/56eca937-06f6-4100-982b-62f3d992b7b4-kube-api-access-tjbqj\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:19.770231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.770150    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:19.770231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.770166    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/56eca937-06f6-4100-982b-62f3d992b7b4-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:19.770231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:19.770182    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/56eca937-06f6-4100-982b-62f3d992b7b4-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:20.386272 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.386175    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" path="/var/lib/kubelet/pods/b40e0f92-ae7c-465e-b236-f7870b4dc4ba/volumes"
Oct 31 04:00:20.389563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.389518    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg" event={"ID":"56eca937-06f6-4100-982b-62f3d992b7b4","Type":"ContainerDied","Data":"bb0411ff0d1a5ceb3b9cd69f2d588b8ccc4baa911db57a5ec735bae4f8586621"}
Oct 31 04:00:20.389828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.389805    5906 scope.go:122] "RemoveContainer" containerID="273d53adce803c7f8ac022e98e84d40df449b588e481e9635cf355af6542c3cb"
Oct 31 04:00:20.389995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.388422    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"
Oct 31 04:00:20.588818 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.588760    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"]
Oct 31 04:00:20.653417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.651722    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-eb5ce91f-5fc6c86bf5-tl5mg"]
Oct 31 04:00:20.746232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:20.736474    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:21.123389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:21.123349    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:21.403661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:21.403564    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:22.380661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:22.380625    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56eca937-06f6-4100-982b-62f3d992b7b4" path="/var/lib/kubelet/pods/56eca937-06f6-4100-982b-62f3d992b7b4/volumes"
Oct 31 04:00:22.988790 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:22.988738    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:23.004419 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:23.004385    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"]
Oct 31 04:00:23.004651 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:23.004627    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" containerID="cri-o://b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa" gracePeriod=30
Oct 31 04:00:23.108251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:23.089273    5906 patch_prober.go:29] interesting pod/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:23.108251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:23.089350    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:23.808414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:23.808385    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:24.449519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.448743    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"]
Oct 31 04:00:24.921459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.921428    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:24.922045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.922023    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:24.922252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.922239    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:24.922344 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.922334    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:24.922781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.922755    5906 cpu_manager.go:414] "containerMap: removing container" podUID="56eca937-06f6-4100-982b-62f3d992b7b4" containerName="maas-api"
Oct 31 04:00:24.923057 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923039    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56eca937-06f6-4100-982b-62f3d992b7b4" containerName="maas-api"
Oct 31 04:00:24.923238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923221    5906 cpu_manager.go:414] "containerMap: removing container" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy"
Oct 31 04:00:24.923362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923347    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy"
Oct 31 04:00:24.923498 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923484    5906 cpu_manager.go:414] "containerMap: removing container" podUID="306169e0-4feb-4fe2-9018-6cdfc3a93eef" containerName="maas-api"
Oct 31 04:00:24.923609 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923595    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="306169e0-4feb-4fe2-9018-6cdfc3a93eef" containerName="maas-api"
Oct 31 04:00:24.923754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923740    5906 cpu_manager.go:414] "containerMap: removing container" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy"
Oct 31 04:00:24.923875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.923861    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy"
Oct 31 04:00:24.989243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.979043    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:25.021310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.021281    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:25.021582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.021564    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="56eca937-06f6-4100-982b-62f3d992b7b4" containerName="maas-api"
Oct 31 04:00:25.021704 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.021694    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy"
Oct 31 04:00:25.021826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.021816    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="306169e0-4feb-4fe2-9018-6cdfc3a93eef" containerName="maas-api"
Oct 31 04:00:25.022783 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:24.935886    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:25.060229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.051271    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"]
Oct 31 04:00:25.060229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.051455    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.122167 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.122128    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.126228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.122463    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.151448 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.151407    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.151842 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.151819    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.152010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.151996    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.152274 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.152257    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.152536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.152519    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4zlt\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.195324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.185315    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.195324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.185422    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.195324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.178985    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-5a5baa-openshift-default-dockercfg-jn95d\""
Oct 31 04:00:25.205230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.196461    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:25.306663 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.306618    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.322269 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.306989    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.322631 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.322596    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.323971 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.323943    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.331259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.309467    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.331526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.324347    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4zlt\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.331672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.331648    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.357393 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.357349    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.358510 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.358483    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.359334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.359311    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.360027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.325032    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.360160 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.358382    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.360287 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.309782    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.360402 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.332091    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.374600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.374526    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.386619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.386572    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.389141 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.389116    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4zlt\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.425925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.425887    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token\") pod \"e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") " pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.435970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.435939    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:25.897361 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:25.897328    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc44d273e_2eb6_41a7_a91e_35905e76ab88.slice/crio-175e20c7c2e52d830bc1ca4f2c5a654b4fbcffcef24c520c6eaffaefb0a8e99a WatchSource:0}: Error finding container 175e20c7c2e52d830bc1ca4f2c5a654b4fbcffcef24c520c6eaffaefb0a8e99a: Status 404 returned error can't find the container with id 175e20c7c2e52d830bc1ca4f2c5a654b4fbcffcef24c520c6eaffaefb0a8e99a
Oct 31 04:00:25.897943 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.897865    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"]
Oct 31 04:00:25.900324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.900295    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:00:25.900426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.900355    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:00:25.900426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.900386    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:00:25.902172 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.902153    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"]
Oct 31 04:00:25.902927 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.902909    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:25.902999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.902929    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:25.903095 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.903083    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40e0f92-ae7c-465e-b236-f7870b4dc4ba" containerName="cleanup"
Oct 31 04:00:25.903167 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.903098    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbe74473-2dc3-4d54-91b6-e93dbb6dee0b" containerName="istio-proxy"
Oct 31 04:00:25.908071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.908051    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:25.975361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.974804    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bbw\" (UniqueName: \"kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:25.975361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.974853    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:25.975361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.974893    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w5-440092-serving-cert\""
Oct 31 04:00:25.975361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.974920    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:25.975361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.974965    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:25.976949 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:25.976925    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"]
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.091796    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bbw\" (UniqueName: \"kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.091857    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.092279    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.092371    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:26.092915    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-serving-cert: secret "maas-api-e2e-shared-b-w5-440092-serving-cert" not found
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:26.092999    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls podName:bdfe7afd-4334-4df4-99c0-7cc0579eddc8 nodeName:}" failed. No retries permitted until 2026-10-31 04:00:26.592968754 +0000 UTC m=+1876.426829060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls") pod "maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" (UID: "bdfe7afd-4334-4df4-99c0-7cc0579eddc8") : secret "maas-api-e2e-shared-b-w5-440092-serving-cert" not found
Oct 31 04:00:26.102280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.094047    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.107971 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.106906    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:26.116047 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.116006    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.201593 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.200004    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bbw\" (UniqueName: \"kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.285184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.285143    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:26.437228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.432149    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:26.659229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.653686    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.674895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.674847    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\") pod \"maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.826078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.826041    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:26.884927 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.884812    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" event={"ID":"c44d273e-2eb6-41a7-a91e-35905e76ab88","Type":"ContainerStarted","Data":"08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"}
Oct 31 04:00:26.884927 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:26.884877    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" event={"ID":"c44d273e-2eb6-41a7-a91e-35905e76ab88","Type":"ContainerStarted","Data":"175e20c7c2e52d830bc1ca4f2c5a654b4fbcffcef24c520c6eaffaefb0a8e99a"}
Oct 31 04:00:27.016261 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.016179    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podStartSLOduration=3.016159132 podStartE2EDuration="3.016159132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:27.010137791 +0000 UTC m=+1876.843998111" watchObservedRunningTime="2026-10-31 04:00:27.016159132 +0000 UTC m=+1876.850019439"
Oct 31 04:00:27.074679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.074642    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"]
Oct 31 04:00:27.074970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.074939    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" containerID="cri-o://b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586" gracePeriod=30
Oct 31 04:00:27.080660 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.080309    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"]
Oct 31 04:00:27.080660 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.080554    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" containerID="cri-o://2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab" gracePeriod=30
Oct 31 04:00:27.084670 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.084642    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"]
Oct 31 04:00:27.084855 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.084833    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" containerID="cri-o://f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d" gracePeriod=30
Oct 31 04:00:27.425389 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:27.425351    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdfe7afd_4334_4df4_99c0_7cc0579eddc8.slice/crio-d8dd283027c8219eaa4b5d29ebe0c27562bb36f71f1c1ea623c3bf2ccb2873d2 WatchSource:0}: Error finding container d8dd283027c8219eaa4b5d29ebe0c27562bb36f71f1c1ea623c3bf2ccb2873d2: Status 404 returned error can't find the container with id d8dd283027c8219eaa4b5d29ebe0c27562bb36f71f1c1ea623c3bf2ccb2873d2
Oct 31 04:00:27.434167 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.434065    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"]
Oct 31 04:00:27.438725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.438689    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:27.470021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.469989    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:27.583443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.583345    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"]
Oct 31 04:00:27.585149 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.585120    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"]
Oct 31 04:00:27.585579 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.585552    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy" containerID="cri-o://1747f294336bfb4fc6acc8e08002098f213cb5c7eaba5bb275024ac8bdd4a7fb" gracePeriod=30
Oct 31 04:00:27.589459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.589435    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"]
Oct 31 04:00:27.589790 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.589765    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" containerID="cri-o://3e1e754575b36de3234b20dfcde4ad406a12ab23617d2569760bb2ee6c6a5672" gracePeriod=30
Oct 31 04:00:27.593564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.593539    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"]
Oct 31 04:00:27.593823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.593800    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" containerID="cri-o://4e66fa13830f352bf5fc52840a342d7c6ea4eaf706bc907cac90454bf5fb3428" gracePeriod=30
Oct 31 04:00:27.895082 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.894984    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" event={"ID":"bdfe7afd-4334-4df4-99c0-7cc0579eddc8","Type":"ContainerStarted","Data":"d8dd283027c8219eaa4b5d29ebe0c27562bb36f71f1c1ea623c3bf2ccb2873d2"}
Oct 31 04:00:27.895277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.895187    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy" containerID="cri-o://9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8" gracePeriod=30
Oct 31 04:00:27.895389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.895370    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:27.896470 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:27.896451    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:00:28.444597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.444564    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 04:00:28.552272 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552238    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552428 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552301    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552428 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552324    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552428 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552393    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552476    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:28.552675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552603    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552678    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552670    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:28.552781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552679    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:28.552781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552768    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552685    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:28.552987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552927    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.552987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552941    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:28.552987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.552970    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo\") pod \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\" (UID: \"fc2f33f4-6678-4542-a9df-0a2a3781c5ab\") "
Oct 31 04:00:28.553824 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.553795    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.553824 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.553826    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.554006 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.553843    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.554006 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.553858    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.554006 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.553872    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.554944 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.554918    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:28.555049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.554955    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:28.555128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.555110    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "kube-api-access-vbzsj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:28.555169 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.555125    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo" pod "fc2f33f4-6678-4542-a9df-0a2a3781c5ab" (UID: "fc2f33f4-6678-4542-a9df-0a2a3781c5ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:28.628071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.628034    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:28.628255 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.628104    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:28.655847 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.655804    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.655847 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.655848    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbzsj\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-kube-api-access-vbzsj\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.656085 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.655866    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.656085 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.655879    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc2f33f4-6678-4542-a9df-0a2a3781c5ab-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:28.905901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.905806    5906 generic.go:356] "Generic (PLEG): container finished" podID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerID="b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa" exitCode=0
Oct 31 04:00:28.906088 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.906032    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" event={"ID":"fc2f33f4-6678-4542-a9df-0a2a3781c5ab","Type":"ContainerDied","Data":"b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa"}
Oct 31 04:00:28.906088 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.906079    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl" event={"ID":"fc2f33f4-6678-4542-a9df-0a2a3781c5ab","Type":"ContainerDied","Data":"3506ceb33581dccac8d5405546b4f1cf151364228812ecb555debb071d085eb8"}
Oct 31 04:00:28.906191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.906105    5906 scope.go:122] "RemoveContainer" containerID="b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa"
Oct 31 04:00:28.906292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.906277    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"
Oct 31 04:00:28.911672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.911638    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" event={"ID":"bdfe7afd-4334-4df4-99c0-7cc0579eddc8","Type":"ContainerStarted","Data":"94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb"}
Oct 31 04:00:28.911926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.911905    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:28.921487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.921470    5906 scope.go:122] "RemoveContainer" containerID="3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"
Oct 31 04:00:28.935788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.934284    5906 scope.go:122] "RemoveContainer" containerID="b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa"
Oct 31 04:00:28.935788 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:28.935679    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa\": container with ID starting with b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa not found: ID does not exist" containerID="b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa"
Oct 31 04:00:28.935788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.935719    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa"} err="failed to get container status \"b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa\": rpc error: code = NotFound desc = could not find container \"b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa\": container with ID starting with b393e85a9978281170026f8cda1098f9b406d05c20561678ef384cd17f869afa not found: ID does not exist"
Oct 31 04:00:28.935788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.935745    5906 scope.go:122] "RemoveContainer" containerID="3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"
Oct 31 04:00:28.936172 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:28.936144    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995\": container with ID starting with 3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995 not found: ID does not exist" containerID="3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"
Oct 31 04:00:28.936282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:28.936182    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995"} err="failed to get container status \"3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995\": rpc error: code = NotFound desc = could not find container \"3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995\": container with ID starting with 3b9d8d6d437925fc33cc8fe9d17f727504d1cd7b61f46531cc0a9d250db79995 not found: ID does not exist"
Oct 31 04:00:29.125226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:29.121126    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" podStartSLOduration=3.049034988 podStartE2EDuration="4.121102053s" totalImagesPullingTime="1.072067065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:29.111684415 +0000 UTC m=+1878.945544721" watchObservedRunningTime="2026-10-31 04:00:29.121102053 +0000 UTC m=+1878.954962361"
Oct 31 04:00:29.340635 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:29.340594    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:29.341056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:29.340980    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:29.399716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:29.397289    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"]
Oct 31 04:00:29.405095 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:29.405027    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-eb5ce91f-openshift-default-6bcc5fdc69-87vgl"]
Oct 31 04:00:29.943379 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:29.943344    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:30.095159 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:30.095125    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:30.260048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:30.248426    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:30.448310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:30.447027    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" path="/var/lib/kubelet/pods/fc2f33f4-6678-4542-a9df-0a2a3781c5ab/volumes"
Oct 31 04:00:30.989174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:30.989138    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:30.989668 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:30.989200    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:31.352484 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:31.352449    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:31.760330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:31.760295    5906 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:31.760495 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:31.760357    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:32.426082 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.426053    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 04:00:32.513999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.513969    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 04:00:32.514321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514297    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.514451 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514432    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.514561 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514465    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.514561 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514486    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.514561 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514518    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.515029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514585    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.515029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514614    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.515029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514816    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.515029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514859    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs\") pod \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\" (UID: \"af7a4364-e1d3-4f50-beb7-3a0c88346fd9\") "
Oct 31 04:00:32.515029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.514843    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.515356 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515084    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.515356 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515305    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.515466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515373    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.515466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515395    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:32.515926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515902    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.516020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515935    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.516020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515949    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.516020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515964    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.516020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.515979    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.518323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.517162    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:32.518323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.517271    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.518323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.517281    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "kube-api-access-h5tdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:32.518323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.518060    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token" pod "af7a4364-e1d3-4f50-beb7-3a0c88346fd9" (UID: "af7a4364-e1d3-4f50-beb7-3a0c88346fd9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:32.518954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.518932    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 04:00:32.617216 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617163    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617368 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617297    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617368 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617332    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617395    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617468    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.617625 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617529    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617712 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617686    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617728    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.617812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617781    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.617995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617856    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.617995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.617924    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.618237 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618179    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.618678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618648    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.618838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618698    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.618838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618761    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.618838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618793    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4\") pod \"4178a3ca-62e0-4436-889d-caa4c2153905\" (UID: \"4178a3ca-62e0-4436-889d-caa4c2153905\") "
Oct 31 04:00:32.619029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618869    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.619029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.618991    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.619029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.619000    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.619259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.619111    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert\") pod \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\" (UID: \"f1f7f3c8-5d5c-4982-aa65-b073260cc330\") "
Oct 31 04:00:32.619503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.619329    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.619820 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.619659    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620341    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620367    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620382    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620396    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620410    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5tdh\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-kube-api-access-h5tdh\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620425    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620439    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af7a4364-e1d3-4f50-beb7-3a0c88346fd9-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.620730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.620453    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.621946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.621546    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:32.621946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.621790    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.622098 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.622077    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.622350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.622240    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.622627 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.622523    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:32.622801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.622755    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.624866 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.624824    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.626993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.626963    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "kube-api-access-2mw5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:32.629908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.629885    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:32.632744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.632702    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:32.632744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.632717    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:32.632744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.632704    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:32.634552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.634526    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4" pod "4178a3ca-62e0-4436-889d-caa4c2153905" (UID: "4178a3ca-62e0-4436-889d-caa4c2153905"). InnerVolumeSpecName "kube-api-access-b8rt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:32.634628 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.634590    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token" pod "f1f7f3c8-5d5c-4982-aa65-b073260cc330" (UID: "f1f7f3c8-5d5c-4982-aa65-b073260cc330"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724130    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724159    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724171    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724186    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4178a3ca-62e0-4436-889d-caa4c2153905-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724199    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724231    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4178a3ca-62e0-4436-889d-caa4c2153905-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724243    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724256    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mw5z\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-kube-api-access-2mw5z\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724419    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724451    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4178a3ca-62e0-4436-889d-caa4c2153905-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724464    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724477    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724491    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f1f7f3c8-5d5c-4982-aa65-b073260cc330-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.724858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.724506    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8rt4\" (UniqueName: \"kubernetes.io/projected/4178a3ca-62e0-4436-889d-caa4c2153905-kube-api-access-b8rt4\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:32.782256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.780644    5906 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 31 04:00:32.782256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.780724    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 31 04:00:32.948503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.948417    5906 generic.go:356] "Generic (PLEG): container finished" podID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerID="2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab" exitCode=0
Oct 31 04:00:32.948503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.948476    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"
Oct 31 04:00:32.948740 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.948543    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" event={"ID":"f1f7f3c8-5d5c-4982-aa65-b073260cc330","Type":"ContainerDied","Data":"2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab"}
Oct 31 04:00:32.948740 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.948593    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz" event={"ID":"f1f7f3c8-5d5c-4982-aa65-b073260cc330","Type":"ContainerDied","Data":"e15b6731e41e14d2c3aa1c19645318b0aec4e7d416798402f58f68116f0107a1"}
Oct 31 04:00:32.948740 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.948621    5906 scope.go:122] "RemoveContainer" containerID="2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab"
Oct 31 04:00:32.952657 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.952625    5906 generic.go:356] "Generic (PLEG): container finished" podID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerID="b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586" exitCode=0
Oct 31 04:00:32.952921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.952730    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" event={"ID":"af7a4364-e1d3-4f50-beb7-3a0c88346fd9","Type":"ContainerDied","Data":"b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586"}
Oct 31 04:00:32.952921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.952777    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd" event={"ID":"af7a4364-e1d3-4f50-beb7-3a0c88346fd9","Type":"ContainerDied","Data":"b0cb30dbf3172564561cff6be13d90fdf4704059dfcafae1980e05395a92063e"}
Oct 31 04:00:32.952921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.952857    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"
Oct 31 04:00:32.961330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.958331    5906 generic.go:356] "Generic (PLEG): container finished" podID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerID="3e1e754575b36de3234b20dfcde4ad406a12ab23617d2569760bb2ee6c6a5672" exitCode=0
Oct 31 04:00:32.961330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.958487    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" event={"ID":"dc248aba-1443-49ca-a0da-4ec62f4f0828","Type":"ContainerDied","Data":"3e1e754575b36de3234b20dfcde4ad406a12ab23617d2569760bb2ee6c6a5672"}
Oct 31 04:00:32.961330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.960089    5906 generic.go:356] "Generic (PLEG): container finished" podID="4178a3ca-62e0-4436-889d-caa4c2153905" containerID="f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d" exitCode=0
Oct 31 04:00:32.961330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.960190    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" event={"ID":"4178a3ca-62e0-4436-889d-caa4c2153905","Type":"ContainerDied","Data":"f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d"}
Oct 31 04:00:32.961330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.960245    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45" event={"ID":"4178a3ca-62e0-4436-889d-caa4c2153905","Type":"ContainerDied","Data":"9921c1765d3396e6012896ea3206f4375df603927a997057f3f99c60722e3f84"}
Oct 31 04:00:32.961330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.960323    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.966982    5906 generic.go:356] "Generic (PLEG): container finished" podID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerID="0087d8aa6f291bfe325aae3f6ffff0c8cb85c62bb30e1cde9c7d8e711919175a" exitCode=0
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.967150    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" event={"ID":"749d4774-58db-40e5-bf02-cf4d37da86dc","Type":"ContainerDied","Data":"0087d8aa6f291bfe325aae3f6ffff0c8cb85c62bb30e1cde9c7d8e711919175a"}
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.967719    5906 scope.go:122] "RemoveContainer" containerID="0087d8aa6f291bfe325aae3f6ffff0c8cb85c62bb30e1cde9c7d8e711919175a"
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.970688    5906 generic.go:356] "Generic (PLEG): container finished" podID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerID="1747f294336bfb4fc6acc8e08002098f213cb5c7eaba5bb275024ac8bdd4a7fb" exitCode=0
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.970800    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" event={"ID":"0300d288-3aec-4909-89c8-45f8ac68cbbd","Type":"ContainerDied","Data":"1747f294336bfb4fc6acc8e08002098f213cb5c7eaba5bb275024ac8bdd4a7fb"}
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.973395    5906 generic.go:356] "Generic (PLEG): container finished" podID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerID="4e66fa13830f352bf5fc52840a342d7c6ea4eaf706bc907cac90454bf5fb3428" exitCode=0
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.973432    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" event={"ID":"b6d0f5af-9f20-4742-bdc2-cc71145288ae","Type":"ContainerDied","Data":"4e66fa13830f352bf5fc52840a342d7c6ea4eaf706bc907cac90454bf5fb3428"}
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.976821    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.976930    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:00:32.977400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:32.976981    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:00:33.016968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.016948    5906 scope.go:122] "RemoveContainer" containerID="2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab"
Oct 31 04:00:33.018267 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:33.018178    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab\": container with ID starting with 2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab not found: ID does not exist" containerID="2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab"
Oct 31 04:00:33.018399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.018361    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab"} err="failed to get container status \"2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab\": rpc error: code = NotFound desc = could not find container \"2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab\": container with ID starting with 2178a9f591b7127734708fbe638bab166585f49274f8267b42ed77bd89a9ceab not found: ID does not exist"
Oct 31 04:00:33.018496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.018481    5906 scope.go:122] "RemoveContainer" containerID="b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586"
Oct 31 04:00:33.019957 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.019927    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 04:00:33.055287 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.055146    5906 scope.go:122] "RemoveContainer" containerID="b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586"
Oct 31 04:00:33.055902 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:33.055771    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586\": container with ID starting with b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586 not found: ID does not exist" containerID="b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586"
Oct 31 04:00:33.055902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.055806    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586"} err="failed to get container status \"b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586\": rpc error: code = NotFound desc = could not find container \"b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586\": container with ID starting with b2a9817f531597a78d41242ab5bd2a79de165f589d13b8f021a3d3cc7ef4f586 not found: ID does not exist"
Oct 31 04:00:33.055902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.055825    5906 scope.go:122] "RemoveContainer" containerID="f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d"
Oct 31 04:00:33.064255 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.064177    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 04:00:33.081630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.079259    5906 scope.go:122] "RemoveContainer" containerID="f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d"
Oct 31 04:00:33.083924 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:33.083818    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d\": container with ID starting with f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d not found: ID does not exist" containerID="f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d"
Oct 31 04:00:33.083924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.083862    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d"} err="failed to get container status \"f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d\": rpc error: code = NotFound desc = could not find container \"f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d\": container with ID starting with f4415a944b00fa9136f16bac799aec77e7ab1bf8c86a2fb0bd4566d65729016d not found: ID does not exist"
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.150748    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.150824    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.150938    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.150966    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151021    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151044    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151070    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151108    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151144    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151200    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151278    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151336    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151398    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151427    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151461    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151513    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151563    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert\") pod \"dc248aba-1443-49ca-a0da-4ec62f4f0828\" (UID: \"dc248aba-1443-49ca-a0da-4ec62f4f0828\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.151607    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp\") pod \"0300d288-3aec-4909-89c8-45f8ac68cbbd\" (UID: \"0300d288-3aec-4909-89c8-45f8ac68cbbd\") "
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.153922    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.154191    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.154516    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.167791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.165144    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:33.194760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.174588    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"]
Oct 31 04:00:33.194760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.174802    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.194760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.175001    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.194760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.175272    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:33.194760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.175493    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.194760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.175808    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.211840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.211776    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.213441 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.213349    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 04:00:33.289676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.289635    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:33.290051 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.290026    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "kube-api-access-hbltq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.290858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.290834    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:33.291126 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.291096    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.291327 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.291307    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token" pod "dc248aba-1443-49ca-a0da-4ec62f4f0828" (UID: "dc248aba-1443-49ca-a0da-4ec62f4f0828"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.291410 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.291371    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.291493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.291460    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "kube-api-access-qqrhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.294351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.294321    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token" pod "0300d288-3aec-4909-89c8-45f8ac68cbbd" (UID: "0300d288-3aec-4909-89c8-45f8ac68cbbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.296188 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.295883    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-s7dpz"]
Oct 31 04:00:33.326892 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.326855    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.326909    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.326943    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.326981    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327006    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327030    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327063    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327130    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327175    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc248aba-1443-49ca-a0da-4ec62f4f0828-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327504 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327191    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0300d288-3aec-4909-89c8-45f8ac68cbbd-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.327504 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.327227    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.428758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.428614    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"]
Oct 31 04:00:33.429433 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.429382    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.429544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.429515    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.429646 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.429588    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.429646 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.429633    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.429844 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.429820    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.429943 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.429888    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.430073 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.430053    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.430162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.430106    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.430162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.430149    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy\") pod \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\" (UID: \"b6d0f5af-9f20-4742-bdc2-cc71145288ae\") "
Oct 31 04:00:33.430842 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.430819    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.431309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.431284    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbltq\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-kube-api-access-hbltq\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.431984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.431829    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.431984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.431852    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.431984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.431866    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.431984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.431880    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc248aba-1443-49ca-a0da-4ec62f4f0828-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.431984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.431894    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqrhp\" (UniqueName: \"kubernetes.io/projected/0300d288-3aec-4909-89c8-45f8ac68cbbd-kube-api-access-qqrhp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.432323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.432131    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.432461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.432435    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.432992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.432967    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 04:00:33.440412 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.435416    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-mhqjd"]
Oct 31 04:00:33.440412 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.439307    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.442527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.442496    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.444870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.444827    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.447328 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.447288    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:33.451322 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.451272    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:33.453532 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.453498    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "kube-api-access-g55lg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.455051 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.454969    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token" pod "b6d0f5af-9f20-4742-bdc2-cc71145288ae" (UID: "b6d0f5af-9f20-4742-bdc2-cc71145288ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534233    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534279    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534295    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g55lg\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-kube-api-access-g55lg\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534311    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534325    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534339    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534352    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534366    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.534998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.534380    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6d0f5af-9f20-4742-bdc2-cc71145288ae-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.541894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.541856    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"]
Oct 31 04:00:33.545770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.545737    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-hnt45"]
Oct 31 04:00:33.635367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635328    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635542 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635430    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635605 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635549    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635684 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635670    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635706    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635745    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635780    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.635901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635859    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.635901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635877    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.636013 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635937    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.636013 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.635986    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data\") pod \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\" (UID: \"a1fb0e3b-e845-472d-8c2b-d6a718e8715e\") "
Oct 31 04:00:33.636238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636172    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:33.636877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636375    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.636877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636605    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.636877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636827    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.636877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636848    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.636877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636863    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.636877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636877    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.637308 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.636890    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.638073 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.638043    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.638173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.638112    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "kube-api-access-62h4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:33.638321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.638250    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:33.638630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.638602    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo" pod "a1fb0e3b-e845-472d-8c2b-d6a718e8715e" (UID: "a1fb0e3b-e845-472d-8c2b-d6a718e8715e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:33.738100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.738015    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.738100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.738047    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.738100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.738058    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62h4f\" (UniqueName: \"kubernetes.io/projected/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-kube-api-access-62h4f\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.738100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.738069    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1fb0e3b-e845-472d-8c2b-d6a718e8715e-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:33.990706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.990621    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"
Oct 31 04:00:33.990706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.990637    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6" event={"ID":"dc248aba-1443-49ca-a0da-4ec62f4f0828","Type":"ContainerDied","Data":"f75623e170fc23cc0c2f85387e4da895061c6e0f0d709b1c019b7365f9c63e12"}
Oct 31 04:00:33.990706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.990699    5906 scope.go:122] "RemoveContainer" containerID="3e1e754575b36de3234b20dfcde4ad406a12ab23617d2569760bb2ee6c6a5672"
Oct 31 04:00:33.994025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.993987    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" event={"ID":"749d4774-58db-40e5-bf02-cf4d37da86dc","Type":"ContainerStarted","Data":"3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef"}
Oct 31 04:00:33.995468 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.995441    5906 generic.go:356] "Generic (PLEG): container finished" podID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerID="9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8" exitCode=0
Oct 31 04:00:33.995580 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.995471    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"
Oct 31 04:00:33.995580 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.995527    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" event={"ID":"a1fb0e3b-e845-472d-8c2b-d6a718e8715e","Type":"ContainerDied","Data":"9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8"}
Oct 31 04:00:33.995697 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.995582    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj" event={"ID":"a1fb0e3b-e845-472d-8c2b-d6a718e8715e","Type":"ContainerDied","Data":"d8f51f28daed6e221f370338be0dd5d78a2c83175fa6b910b9cecb9bebad7267"}
Oct 31 04:00:33.997075 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.997056    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"
Oct 31 04:00:33.997186 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.997054    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw" event={"ID":"0300d288-3aec-4909-89c8-45f8ac68cbbd","Type":"ContainerDied","Data":"aca8be811513f8a2f36fff39b55592c000e85c44778beec86d7f4518436a859c"}
Oct 31 04:00:33.999250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.998737    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"
Oct 31 04:00:33.999250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:33.998753    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c" event={"ID":"b6d0f5af-9f20-4742-bdc2-cc71145288ae","Type":"ContainerDied","Data":"f6737e5dbb20a8d0f85422f6a17d9e1512c63f27435f7370e6d184ddc2913487"}
Oct 31 04:00:34.003475 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.003454    5906 scope.go:122] "RemoveContainer" containerID="9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8"
Oct 31 04:00:34.015877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.015780    5906 scope.go:122] "RemoveContainer" containerID="9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8"
Oct 31 04:00:34.016199 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:34.016168    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8\": container with ID starting with 9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8 not found: ID does not exist" containerID="9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8"
Oct 31 04:00:34.017330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.017293    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8"} err="failed to get container status \"9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8\": rpc error: code = NotFound desc = could not find container \"9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8\": container with ID starting with 9de3d0895b0c0539d4691e3d361ed25af63d68d7c8946930218930896bc402d8 not found: ID does not exist"
Oct 31 04:00:34.017330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.017331    5906 scope.go:122] "RemoveContainer" containerID="1747f294336bfb4fc6acc8e08002098f213cb5c7eaba5bb275024ac8bdd4a7fb"
Oct 31 04:00:34.045686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.045660    5906 scope.go:122] "RemoveContainer" containerID="4e66fa13830f352bf5fc52840a342d7c6ea4eaf706bc907cac90454bf5fb3428"
Oct 31 04:00:34.052116 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.052042    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 04:00:34.053058 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.053029    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:34.053180 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.053087    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:34.158505 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.158468    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"]
Oct 31 04:00:34.163620 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.163593    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-cj6kj"]
Oct 31 04:00:34.188236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.188191    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:34.273486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.273452    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"]
Oct 31 04:00:34.279737 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.279706    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-hxhjw"]
Oct 31 04:00:34.551286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.550759    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:34.638281 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.637853    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" path="/var/lib/kubelet/pods/0300d288-3aec-4909-89c8-45f8ac68cbbd/volumes"
Oct 31 04:00:34.640608 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.639092    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" path="/var/lib/kubelet/pods/4178a3ca-62e0-4436-889d-caa4c2153905/volumes"
Oct 31 04:00:34.640608 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.639911    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" path="/var/lib/kubelet/pods/a1fb0e3b-e845-472d-8c2b-d6a718e8715e/volumes"
Oct 31 04:00:34.641461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.641415    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" path="/var/lib/kubelet/pods/af7a4364-e1d3-4f50-beb7-3a0c88346fd9/volumes"
Oct 31 04:00:34.642182 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.642157    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" path="/var/lib/kubelet/pods/f1f7f3c8-5d5c-4982-aa65-b073260cc330/volumes"
Oct 31 04:00:34.682302 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.682265    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"]
Oct 31 04:00:34.698702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.698667    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-7h72c"]
Oct 31 04:00:34.704114 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.704084    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"]
Oct 31 04:00:34.705688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.705368    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy"
Oct 31 04:00:34.705895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.705874    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy"
Oct 31 04:00:34.706012 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.705998    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy"
Oct 31 04:00:34.706113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.706097    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy"
Oct 31 04:00:34.706305 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.706289    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy"
Oct 31 04:00:34.706395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.706383    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy"
Oct 31 04:00:34.706516 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.706504    5906 cpu_manager.go:414] "containerMap: removing container" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy"
Oct 31 04:00:34.706610 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.706598    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy"
Oct 31 04:00:34.706925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.706909    5906 cpu_manager.go:414] "containerMap: removing container" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy"
Oct 31 04:00:34.707029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707018    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy"
Oct 31 04:00:34.707140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707129    5906 cpu_manager.go:414] "containerMap: removing container" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy"
Oct 31 04:00:34.707238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707225    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy"
Oct 31 04:00:34.707324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707314    5906 cpu_manager.go:414] "containerMap: removing container" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy"
Oct 31 04:00:34.707407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707397    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy"
Oct 31 04:00:34.707494 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707485    5906 cpu_manager.go:414] "containerMap: removing container" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy"
Oct 31 04:00:34.707571 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707561    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy"
Oct 31 04:00:34.707660 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707651    5906 cpu_manager.go:414] "containerMap: removing container" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy"
Oct 31 04:00:34.707730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707720    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy"
Oct 31 04:00:34.707991 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.707974    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy"
Oct 31 04:00:34.708100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708089    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" containerName="istio-proxy"
Oct 31 04:00:34.708182 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708173    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="4178a3ca-62e0-4436-889d-caa4c2153905" containerName="istio-proxy"
Oct 31 04:00:34.708302 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708283    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="0300d288-3aec-4909-89c8-45f8ac68cbbd" containerName="istio-proxy"
Oct 31 04:00:34.708397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708388    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="af7a4364-e1d3-4f50-beb7-3a0c88346fd9" containerName="istio-proxy"
Oct 31 04:00:34.708485 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708476    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1f7f3c8-5d5c-4982-aa65-b073260cc330" containerName="istio-proxy"
Oct 31 04:00:34.708560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708551    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1fb0e3b-e845-472d-8c2b-d6a718e8715e" containerName="istio-proxy"
Oct 31 04:00:34.708637 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.708629    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" containerName="istio-proxy"
Oct 31 04:00:34.720226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.718290    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"]
Oct 31 04:00:34.720226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.718461    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:34.816316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.801023    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-5a5baa\""
Oct 31 04:00:34.816316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.801158    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-5a5baa-dockercfg-pc7fc\""
Oct 31 04:00:34.816316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.803915    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"]
Oct 31 04:00:34.816316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.814985    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:34.816316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.815139    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hbhz\" (UniqueName: \"kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz\") pod \"payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:34.901768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.899489    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc2f33f4-6678-4542-a9df-0a2a3781c5ab" containerName="istio-proxy"
Oct 31 04:00:34.906318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.905693    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.919479    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.919618    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.919655    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phpst\" (UniqueName: \"kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst\") pod \"payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.919690    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbhz\" (UniqueName: \"kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz\") pod \"payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.920836    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.929216    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 04:00:34.950229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:34.936016    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:00:35.079285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.061451    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:35.079285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.061518    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:35.123417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.087419    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:35.123417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.087485    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phpst\" (UniqueName: \"kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst\") pod \"payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:35.123417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.089328    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:35.126321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.124379    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hbhz\" (UniqueName: \"kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz\") pod \"payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:35.137505 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.136884    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"]
Oct 31 04:00:35.175782 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.163270    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-tq9t6"]
Oct 31 04:00:35.175782 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.171687    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"]
Oct 31 04:00:35.204429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.204393    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phpst\" (UniqueName: \"kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst\") pod \"payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:35.270695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.270656    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:35.467893 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.467562    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:35.669894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.669864    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"]
Oct 31 04:00:35.671473 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:35.671446    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2dccd3f_32eb_4725_8957_7cd1d61fe22a.slice/crio-dd18c6ad3208071c8d0cf7be4826ccdd904a37556285a8d447ad92cd92a90905 WatchSource:0}: Error finding container dd18c6ad3208071c8d0cf7be4826ccdd904a37556285a8d447ad92cd92a90905: Status 404 returned error can't find the container with id dd18c6ad3208071c8d0cf7be4826ccdd904a37556285a8d447ad92cd92a90905
Oct 31 04:00:35.871649 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:00:35.871609    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2741d67f_f693_4bb4_99fe_c169b380e663.slice/crio-e141a9ef1b7f158351927a42346744dc4b5a6af7b271128f4ced1daeb837ac4a WatchSource:0}: Error finding container e141a9ef1b7f158351927a42346744dc4b5a6af7b271128f4ced1daeb837ac4a: Status 404 returned error can't find the container with id e141a9ef1b7f158351927a42346744dc4b5a6af7b271128f4ced1daeb837ac4a
Oct 31 04:00:35.872260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:35.872226    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"]
Oct 31 04:00:36.066281 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.062795    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:36.066281 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.062857    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:36.198514 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.198419    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:36.206544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.206508    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2" event={"ID":"2741d67f-f693-4bb4-99fe-c169b380e663","Type":"ContainerStarted","Data":"4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed"}
Oct 31 04:00:36.206741 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.206576    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:36.206741 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.206599    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2" event={"ID":"2741d67f-f693-4bb4-99fe-c169b380e663","Type":"ContainerStarted","Data":"e141a9ef1b7f158351927a42346744dc4b5a6af7b271128f4ced1daeb837ac4a"}
Oct 31 04:00:36.208170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.208135    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k" event={"ID":"f2dccd3f-32eb-4725-8957-7cd1d61fe22a","Type":"ContainerStarted","Data":"b7213fb3dad5f79c99cbd5ab3a8894459c4297383caf28a6e59a2eb1fa412828"}
Oct 31 04:00:36.208286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.208186    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k" event={"ID":"f2dccd3f-32eb-4725-8957-7cd1d61fe22a","Type":"ContainerStarted","Data":"dd18c6ad3208071c8d0cf7be4826ccdd904a37556285a8d447ad92cd92a90905"}
Oct 31 04:00:36.208399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.208381    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:36.343243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.342732    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2" podStartSLOduration=2.342711813 podStartE2EDuration="2.342711813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:36.341638517 +0000 UTC m=+1886.175498826" watchObservedRunningTime="2026-10-31 04:00:36.342711813 +0000 UTC m=+1886.176572119"
Oct 31 04:00:36.387380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.386660    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6d0f5af-9f20-4742-bdc2-cc71145288ae" path="/var/lib/kubelet/pods/b6d0f5af-9f20-4742-bdc2-cc71145288ae/volumes"
Oct 31 04:00:36.388381 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:36.387797    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc248aba-1443-49ca-a0da-4ec62f4f0828" path="/var/lib/kubelet/pods/dc248aba-1443-49ca-a0da-4ec62f4f0828/volumes"
Oct 31 04:00:36.486080 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:00:36.485958    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:00:37.061072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:37.060617    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:37.061072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:37.060682    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:38.066424 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:38.066380    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:38.066944 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:38.066459    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:39.052717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:39.052677    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:39.052971 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:39.052944    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:40.060436 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:40.060395    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:00:40.061051 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:40.061016    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:00:40.788575 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:40.788536    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:41.057282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:41.057194    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 04:00:41.058120 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:41.058100    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 04:00:41.286616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:41.286567    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k" podStartSLOduration=7.286552321 podStartE2EDuration="7.286552321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:36.482121532 +0000 UTC m=+1886.315981838" watchObservedRunningTime="2026-10-31 04:00:41.286552321 +0000 UTC m=+1891.120412641"
Oct 31 04:00:45.223791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:45.223759    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:47.218478 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:47.218444    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:00:47.219116 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:47.218538    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:00:49.751405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:49.751303    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:54.258484 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:54.258446    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:56.574328 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.574290    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"]
Oct 31 04:00:56.574766 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.574506    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy" containerID="cri-o://0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd" gracePeriod=30
Oct 31 04:00:56.626345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.626306    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"]
Oct 31 04:00:56.626630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.626591    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" containerID="cri-o://7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654" gracePeriod=30
Oct 31 04:00:56.630486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.630453    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"]
Oct 31 04:00:56.630759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.630719    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" containerID="cri-o://ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d" gracePeriod=30
Oct 31 04:00:56.678529 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.678497    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"]
Oct 31 04:00:56.678868 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.678732    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" containerID="cri-o://4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89" gracePeriod=30
Oct 31 04:00:56.782813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.782778    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"]
Oct 31 04:00:56.783085 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.783053    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" containerID="cri-o://ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb" gracePeriod=30
Oct 31 04:00:56.787226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.787178    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"]
Oct 31 04:00:56.787501 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.787471    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy" containerID="cri-o://7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c" gracePeriod=30
Oct 31 04:00:56.801642 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.801606    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"]
Oct 31 04:00:56.801895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.801865    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy" containerID="cri-o://232c82461e381aa65418c6dc7d7dbf94d235e0bdb8de1d99aff5bc6281ed8b79" gracePeriod=30
Oct 31 04:00:56.805335 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.805305    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"]
Oct 31 04:00:56.805580 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:56.805551    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy" containerID="cri-o://a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd" gracePeriod=30
Oct 31 04:00:57.681043 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:57.681005    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:57.681455 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:57.681077    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:58.819128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:58.819090    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:00:58.948321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:58.948277    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:58.953516 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:58.948595    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:59.137258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:59.137146    5906 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:59.137258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:59.137215    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:59.579589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:00:59.579555    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:00.625940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:00.625906    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:00.770937 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:00.770901    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:01.279756 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:01.279718    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:01.374315 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:01.374276    5906 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:01:01.374479 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:01.374346    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:01:02.123472 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.123402    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 04:01:02.127803 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.127774    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 04:01:02.156252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.151076    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 04:01:02.156252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.153193    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 04:01:02.232888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.232859    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 04:01:02.242816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.242768    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.243183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.243140    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.243418 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.243401    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.244714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.243159    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.244803 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.243674    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.244902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.244882    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.245031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245017    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.245178 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245166    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.245314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245300    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.245463 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245315    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.245542 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245453    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.245645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245616    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.245722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245709    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.245830 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245818    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.245984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245964    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.246110 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246096    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245827    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.245921    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246233    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246341    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246378    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246427    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246459    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246492    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246524    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246552    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.246933    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247087    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247388    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247450    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247485    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247541    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247572    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247613    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247675    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247705    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247744    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247787    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247838    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247866    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy\") pod \"00104874-4cc9-4129-8923-0e301106d24e\" (UID: \"00104874-4cc9-4129-8923-0e301106d24e\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247906    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq\") pod \"3fb16942-2466-4501-8b09-eb90508d79c2\" (UID: \"3fb16942-2466-4501-8b09-eb90508d79c2\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.247942    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.248045    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs\") pod \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\" (UID: \"a79d3196-a04c-40a0-9a08-f1cb72a4a870\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.248077    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy\") pod \"234cd54a-a721-4edb-834a-01d4a9c72042\" (UID: \"234cd54a-a721-4edb-834a-01d4a9c72042\") "
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249138    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249160    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249175    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249190    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249221    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249237    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249260    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.249564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249274    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.251935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.249977    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "kube-api-access-jzl7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.251935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.250309    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.251935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.250546    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.252096 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.252064    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.253286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.252366    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.253286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.252683    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.253286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.253024    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.253286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.253066    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.253973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.253597    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.253973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.253736    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.254694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.254294    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.254694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.254472    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.254694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.254670    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.255005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.254982    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.256674 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.256639    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.256859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.256828    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.261362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.261321    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.261362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.261347    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "kube-api-access-p2jxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.261526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.261376    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.261808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.261678    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.261808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.261753    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token" pod "00104874-4cc9-4129-8923-0e301106d24e" (UID: "00104874-4cc9-4129-8923-0e301106d24e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.261808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.261778    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "kube-api-access-gbgcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.262084 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.262063    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.262374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.262093    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.262374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.262124    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn" pod "234cd54a-a721-4edb-834a-01d4a9c72042" (UID: "234cd54a-a721-4edb-834a-01d4a9c72042"). InnerVolumeSpecName "kube-api-access-sr6vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.262374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.262129    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy" pod "3fb16942-2466-4501-8b09-eb90508d79c2" (UID: "3fb16942-2466-4501-8b09-eb90508d79c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.262374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.262345    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.264240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.264199    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo" pod "a79d3196-a04c-40a0-9a08-f1cb72a4a870" (UID: "a79d3196-a04c-40a0-9a08-f1cb72a4a870"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.272134 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.272110    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 04:01:02.275849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.275829    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 04:01:02.349713 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349676    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.349918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349740    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.349918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349771    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.349918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349805    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.349918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349850    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.349918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349901    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349937    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349969    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349999    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.349998    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350035    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350067    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350105    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350153    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.350249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350236    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350294    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350314    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350342    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350373    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350396    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350406    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350473    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350510    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350597    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350622    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.350732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350711    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350747    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350799    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy\") pod \"82e25961-bd59-441a-9b0a-855dee1f6f47\" (UID: \"82e25961-bd59-441a-9b0a-855dee1f6f47\") "
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350848    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token\") pod \"2e707f40-eb62-40b1-a9da-f302f66c83ee\" (UID: \"2e707f40-eb62-40b1-a9da-f302f66c83ee\") "
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350868    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350886    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.350935    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data\") pod \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\" (UID: \"b96ac7cb-59ab-4824-9c9f-6439c5d9b113\") "
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351027    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.351343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351297    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351520    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351671    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351691    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351696    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr6vn\" (UniqueName: \"kubernetes.io/projected/234cd54a-a721-4edb-834a-01d4a9c72042-kube-api-access-sr6vn\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351735    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzl7b\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-kube-api-access-jzl7b\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351740    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351754    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00104874-4cc9-4129-8923-0e301106d24e-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351769    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.351786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351783    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/234cd54a-a721-4edb-834a-01d4a9c72042-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351805    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351826    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fb16942-2466-4501-8b09-eb90508d79c2-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351848    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00104874-4cc9-4129-8923-0e301106d24e-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351869    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/234cd54a-a721-4edb-834a-01d4a9c72042-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351890    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351911    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351930    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351949    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fb16942-2466-4501-8b09-eb90508d79c2-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351965    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351980    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351993    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352007    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352020    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352033    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00104874-4cc9-4129-8923-0e301106d24e-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352048    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2jxq\" (UniqueName: \"kubernetes.io/projected/3fb16942-2466-4501-8b09-eb90508d79c2-kube-api-access-p2jxq\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352063    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352077    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352091    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/234cd54a-a721-4edb-834a-01d4a9c72042-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352107    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00104874-4cc9-4129-8923-0e301106d24e-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352120    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352134    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352147    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352161    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82e25961-bd59-441a-9b0a-855dee1f6f47-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352174    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a79d3196-a04c-40a0-9a08-f1cb72a4a870-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352188    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbgcp\" (UniqueName: \"kubernetes.io/projected/a79d3196-a04c-40a0-9a08-f1cb72a4a870-kube-api-access-gbgcp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352218    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352233    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e707f40-eb62-40b1-a9da-f302f66c83ee-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352246    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.352244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352260    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.353727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.352273    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fb16942-2466-4501-8b09-eb90508d79c2-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.353727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.351782    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.353727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.353051    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.354406 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.354371    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.354565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.354537    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.354881 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.354747    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.354881 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.354874    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.355598 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.355574    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "kube-api-access-74wfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.356562 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.356539    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.356562 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.356552    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.356706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.356636    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "kube-api-access-9d9hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.356775 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.356704    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "kube-api-access-7dfq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.356827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.356808    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.357497 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.357404    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.357497 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.357475    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo" pod "2e707f40-eb62-40b1-a9da-f302f66c83ee" (UID: "2e707f40-eb62-40b1-a9da-f302f66c83ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.357795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.357729    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo" pod "82e25961-bd59-441a-9b0a-855dee1f6f47" (UID: "82e25961-bd59-441a-9b0a-855dee1f6f47"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.359141 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.359119    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.359431 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.359391    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy" pod "b96ac7cb-59ab-4824-9c9f-6439c5d9b113" (UID: "b96ac7cb-59ab-4824-9c9f-6439c5d9b113"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.454689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454653    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454693    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454712    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454726    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454741    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454755    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454768    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454783    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e707f40-eb62-40b1-a9da-f302f66c83ee-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454796    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d9hr\" (UniqueName: \"kubernetes.io/projected/2e707f40-eb62-40b1-a9da-f302f66c83ee-kube-api-access-9d9hr\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454812    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454825    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454838    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454851    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454864    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454878    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.454894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454891    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74wfz\" (UniqueName: \"kubernetes.io/projected/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-kube-api-access-74wfz\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.455390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454905    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b96ac7cb-59ab-4824-9c9f-6439c5d9b113-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.455390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454918    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dfq8\" (UniqueName: \"kubernetes.io/projected/82e25961-bd59-441a-9b0a-855dee1f6f47-kube-api-access-7dfq8\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.455390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.454932    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82e25961-bd59-441a-9b0a-855dee1f6f47-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.479346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.479310    5906 generic.go:356] "Generic (PLEG): container finished" podID="234cd54a-a721-4edb-834a-01d4a9c72042" containerID="7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654" exitCode=0
Oct 31 04:01:02.479525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.479369    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"
Oct 31 04:01:02.479525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.479397    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" event={"ID":"234cd54a-a721-4edb-834a-01d4a9c72042","Type":"ContainerDied","Data":"7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654"}
Oct 31 04:01:02.479525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.479457    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5" event={"ID":"234cd54a-a721-4edb-834a-01d4a9c72042","Type":"ContainerDied","Data":"46ebdb2439665ff29bd488a700bd79a4927c3e62539ececf31462c718567cfa4"}
Oct 31 04:01:02.479525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.479490    5906 scope.go:122] "RemoveContainer" containerID="7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654"
Oct 31 04:01:02.481457 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.481435    5906 generic.go:356] "Generic (PLEG): container finished" podID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerID="a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd" exitCode=0
Oct 31 04:01:02.481589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.481489    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"
Oct 31 04:01:02.481589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.481532    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" event={"ID":"2e707f40-eb62-40b1-a9da-f302f66c83ee","Type":"ContainerDied","Data":"a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd"}
Oct 31 04:01:02.481589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.481584    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn" event={"ID":"2e707f40-eb62-40b1-a9da-f302f66c83ee","Type":"ContainerDied","Data":"b5e109bbaf166ee026d429f9ffd926849e5c6b16be75c053691bb9c338acb1e9"}
Oct 31 04:01:02.483444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.483419    5906 generic.go:356] "Generic (PLEG): container finished" podID="3fb16942-2466-4501-8b09-eb90508d79c2" containerID="ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d" exitCode=0
Oct 31 04:01:02.483601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.483462    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"
Oct 31 04:01:02.483601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.483479    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" event={"ID":"3fb16942-2466-4501-8b09-eb90508d79c2","Type":"ContainerDied","Data":"ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d"}
Oct 31 04:01:02.483601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.483526    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr" event={"ID":"3fb16942-2466-4501-8b09-eb90508d79c2","Type":"ContainerDied","Data":"714e0e80c64e6a979f13643f7843b9204b576293fe98b3a3d5af589e759f85dc"}
Oct 31 04:01:02.485053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.485031    5906 generic.go:356] "Generic (PLEG): container finished" podID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerID="4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89" exitCode=0
Oct 31 04:01:02.485150 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.485060    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"
Oct 31 04:01:02.485150 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.485112    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" event={"ID":"a79d3196-a04c-40a0-9a08-f1cb72a4a870","Type":"ContainerDied","Data":"4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89"}
Oct 31 04:01:02.485307 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.485151    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89" event={"ID":"a79d3196-a04c-40a0-9a08-f1cb72a4a870","Type":"ContainerDied","Data":"135e4d8c14c388d571530f867a5ef3f33555f9a58efcbda33aa6cc26f4fc6fcf"}
Oct 31 04:01:02.486558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.486537    5906 generic.go:356] "Generic (PLEG): container finished" podID="00104874-4cc9-4129-8923-0e301106d24e" containerID="0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd" exitCode=0
Oct 31 04:01:02.486647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.486573    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"
Oct 31 04:01:02.486647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.486613    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" event={"ID":"00104874-4cc9-4129-8923-0e301106d24e","Type":"ContainerDied","Data":"0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd"}
Oct 31 04:01:02.486737 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.486652    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8" event={"ID":"00104874-4cc9-4129-8923-0e301106d24e","Type":"ContainerDied","Data":"7571d085d856656b461aec7e9ac6b9d45428f4964483dea7df1e50a2bf8ec1e8"}
Oct 31 04:01:02.488253 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.488232    5906 generic.go:356] "Generic (PLEG): container finished" podID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerID="232c82461e381aa65418c6dc7d7dbf94d235e0bdb8de1d99aff5bc6281ed8b79" exitCode=0
Oct 31 04:01:02.488361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.488258    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" event={"ID":"e41e8673-22e9-4fd9-b3b2-43154730fc18","Type":"ContainerDied","Data":"232c82461e381aa65418c6dc7d7dbf94d235e0bdb8de1d99aff5bc6281ed8b79"}
Oct 31 04:01:02.490191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.490169    5906 generic.go:356] "Generic (PLEG): container finished" podID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerID="ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb" exitCode=0
Oct 31 04:01:02.490377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.490196    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" event={"ID":"82e25961-bd59-441a-9b0a-855dee1f6f47","Type":"ContainerDied","Data":"ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb"}
Oct 31 04:01:02.490493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.490410    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn" event={"ID":"82e25961-bd59-441a-9b0a-855dee1f6f47","Type":"ContainerDied","Data":"2b4cc32a69d87cf67f8fc275c4011be318e275053835e89cdb6171d5df6ef2a0"}
Oct 31 04:01:02.490493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.490274    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"
Oct 31 04:01:02.492381 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.492359    5906 generic.go:356] "Generic (PLEG): container finished" podID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerID="7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c" exitCode=0
Oct 31 04:01:02.492481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.492384    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" event={"ID":"b96ac7cb-59ab-4824-9c9f-6439c5d9b113","Type":"ContainerDied","Data":"7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c"}
Oct 31 04:01:02.492481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.492448    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl" event={"ID":"b96ac7cb-59ab-4824-9c9f-6439c5d9b113","Type":"ContainerDied","Data":"01cbba417705d36a114f0fe917fe47135517d9b298d184caa149b884e0368755"}
Oct 31 04:01:02.492481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.492409    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"
Oct 31 04:01:02.495840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.495819    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 04:01:02.496162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.496148    5906 scope.go:122] "RemoveContainer" containerID="7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654"
Oct 31 04:01:02.496494 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.496473    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654\": container with ID starting with 7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654 not found: ID does not exist" containerID="7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654"
Oct 31 04:01:02.496595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.496500    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654"} err="failed to get container status \"7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654\": rpc error: code = NotFound desc = could not find container \"7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654\": container with ID starting with 7530108162eb4bb9d20be2393630a59e3f51b13646e7aa397a653c75589e2654 not found: ID does not exist"
Oct 31 04:01:02.496595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.496518    5906 scope.go:122] "RemoveContainer" containerID="a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd"
Oct 31 04:01:02.507034 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.507004    5906 scope.go:122] "RemoveContainer" containerID="a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd"
Oct 31 04:01:02.507377 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.507350    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd\": container with ID starting with a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd not found: ID does not exist" containerID="a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd"
Oct 31 04:01:02.507456 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.507378    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd"} err="failed to get container status \"a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd\": rpc error: code = NotFound desc = could not find container \"a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd\": container with ID starting with a67aa8db98fd8b8aa37c86d695352803c3f014cc2a93820f5d9e92d2dab689cd not found: ID does not exist"
Oct 31 04:01:02.507456 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.507398    5906 scope.go:122] "RemoveContainer" containerID="ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d"
Oct 31 04:01:02.517581 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.517558    5906 scope.go:122] "RemoveContainer" containerID="ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d"
Oct 31 04:01:02.517914 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.517887    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d\": container with ID starting with ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d not found: ID does not exist" containerID="ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d"
Oct 31 04:01:02.518025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.517921    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d"} err="failed to get container status \"ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d\": rpc error: code = NotFound desc = could not find container \"ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d\": container with ID starting with ec2032b33a205b4678055d43ed7f0cc4369f1e2626b1019c891424cd8c025a5d not found: ID does not exist"
Oct 31 04:01:02.518025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.517945    5906 scope.go:122] "RemoveContainer" containerID="4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89"
Oct 31 04:01:02.528091 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.528064    5906 scope.go:122] "RemoveContainer" containerID="4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89"
Oct 31 04:01:02.528404 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.528385    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89\": container with ID starting with 4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89 not found: ID does not exist" containerID="4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89"
Oct 31 04:01:02.528463 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.528417    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89"} err="failed to get container status \"4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89\": rpc error: code = NotFound desc = could not find container \"4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89\": container with ID starting with 4f5fd0e43e502fc6109e444a8910675bbd5566cace9fe9dc8325c6ffeabe5b89 not found: ID does not exist"
Oct 31 04:01:02.528463 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.528439    5906 scope.go:122] "RemoveContainer" containerID="0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd"
Oct 31 04:01:02.549530 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.549026    5906 scope.go:122] "RemoveContainer" containerID="0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd"
Oct 31 04:01:02.551773 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.551730    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd\": container with ID starting with 0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd not found: ID does not exist" containerID="0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd"
Oct 31 04:01:02.551925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.551774    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd"} err="failed to get container status \"0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd\": rpc error: code = NotFound desc = could not find container \"0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd\": container with ID starting with 0454c075036cef421dcb800ef45d6ef85ca8c7196eb0f57000f8b1eaee99ebdd not found: ID does not exist"
Oct 31 04:01:02.551925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.551800    5906 scope.go:122] "RemoveContainer" containerID="ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb"
Oct 31 04:01:02.566468 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.566402    5906 scope.go:122] "RemoveContainer" containerID="9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"
Oct 31 04:01:02.580189 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.580160    5906 scope.go:122] "RemoveContainer" containerID="ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb"
Oct 31 04:01:02.580715 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.580688    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb\": container with ID starting with ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb not found: ID does not exist" containerID="ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb"
Oct 31 04:01:02.580835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.580728    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb"} err="failed to get container status \"ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb\": rpc error: code = NotFound desc = could not find container \"ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb\": container with ID starting with ef7023d37064371bdac32d0ed7cb19e44b8b933baaf5108ba0639f98991c73eb not found: ID does not exist"
Oct 31 04:01:02.580835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.580761    5906 scope.go:122] "RemoveContainer" containerID="9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"
Oct 31 04:01:02.581149 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.581124    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317\": container with ID starting with 9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317 not found: ID does not exist" containerID="9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"
Oct 31 04:01:02.581289 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.581159    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317"} err="failed to get container status \"9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317\": rpc error: code = NotFound desc = could not find container \"9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317\": container with ID starting with 9d576cbb0f7ec34a108cceb4a8adf431ac76cdd50112a0cb805198d571f46317 not found: ID does not exist"
Oct 31 04:01:02.581289 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.581184    5906 scope.go:122] "RemoveContainer" containerID="7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c"
Oct 31 04:01:02.592140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.592115    5906 scope.go:122] "RemoveContainer" containerID="7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c"
Oct 31 04:01:02.592571 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:02.592540    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c\": container with ID starting with 7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c not found: ID does not exist" containerID="7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c"
Oct 31 04:01:02.592678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.592579    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c"} err="failed to get container status \"7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c\": rpc error: code = NotFound desc = could not find container \"7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c\": container with ID starting with 7b882256e0b517f073deb360e94b7eb45e68e72abf0d6401cce399071778965c not found: ID does not exist"
Oct 31 04:01:02.662797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.662616    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.670822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.663017    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.670822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.663158    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.670822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.663196    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.670822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.663243    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.675465 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.663295    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.675619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.663992    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.675619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.670793    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.677763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.675913    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.677763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.677706    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:01:02.678062 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.678038    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.678824 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.678241    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy\") pod \"e41e8673-22e9-4fd9-b3b2-43154730fc18\" (UID: \"e41e8673-22e9-4fd9-b3b2-43154730fc18\") "
Oct 31 04:01:02.678824 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.678386    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.679151 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.679125    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.679293 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.679153    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.679293 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.679169    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e41e8673-22e9-4fd9-b3b2-43154730fc18-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.679293 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.679183    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.683260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.680490    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.683832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.683800    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:01:02.688385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.688355    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:01:02.689660 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.689630    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.690950 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.690923    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8" pod "e41e8673-22e9-4fd9-b3b2-43154730fc18" (UID: "e41e8673-22e9-4fd9-b3b2-43154730fc18"). InnerVolumeSpecName "kube-api-access-7qtk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:01:02.780434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.780387    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.780434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.780433    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qtk8\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-kube-api-access-7qtk8\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.780434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.780444    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.780434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.780453    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:02.780434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:02.780462    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e41e8673-22e9-4fd9-b3b2-43154730fc18-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:01:03.183936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.183779    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"]
Oct 31 04:01:03.187709 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.187682    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-h26bl"]
Oct 31 04:01:03.299133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.299099    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"]
Oct 31 04:01:03.307477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.307445    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-thfg8"]
Oct 31 04:01:03.420359 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.420318    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"]
Oct 31 04:01:03.424655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.424622    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8lgfn"]
Oct 31 04:01:03.505221 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.505183    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh" event={"ID":"e41e8673-22e9-4fd9-b3b2-43154730fc18","Type":"ContainerDied","Data":"da35decce63ff9028fcaab37509697b49770c4e2a8a601a700281957e3b65b32"}
Oct 31 04:01:03.505417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.505197    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"
Oct 31 04:01:03.505417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.505259    5906 scope.go:122] "RemoveContainer" containerID="232c82461e381aa65418c6dc7d7dbf94d235e0bdb8de1d99aff5bc6281ed8b79"
Oct 31 04:01:03.540727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.540688    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"]
Oct 31 04:01:03.543794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.543763    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-rdrcr"]
Oct 31 04:01:03.805829 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.805739    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"]
Oct 31 04:01:03.818390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.818355    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8ddhn"]
Oct 31 04:01:03.896097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.896063    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:03.941306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:03.941272    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"]
Oct 31 04:01:04.018334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.018291    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-8gp89"]
Oct 31 04:01:04.058254 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.058149    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"]
Oct 31 04:01:04.062613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.062583    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-bz9x5"]
Oct 31 04:01:04.176176 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.176145    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"]
Oct 31 04:01:04.178876 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.178851    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-bshxh"]
Oct 31 04:01:04.354814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.354734    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00104874-4cc9-4129-8923-0e301106d24e" path="/var/lib/kubelet/pods/00104874-4cc9-4129-8923-0e301106d24e/volumes"
Oct 31 04:01:04.355301 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.355285    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" path="/var/lib/kubelet/pods/234cd54a-a721-4edb-834a-01d4a9c72042/volumes"
Oct 31 04:01:04.355736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.355722    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" path="/var/lib/kubelet/pods/2e707f40-eb62-40b1-a9da-f302f66c83ee/volumes"
Oct 31 04:01:04.356502 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.356487    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" path="/var/lib/kubelet/pods/3fb16942-2466-4501-8b09-eb90508d79c2/volumes"
Oct 31 04:01:04.356914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.356901    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" path="/var/lib/kubelet/pods/82e25961-bd59-441a-9b0a-855dee1f6f47/volumes"
Oct 31 04:01:04.357341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.357328    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" path="/var/lib/kubelet/pods/a79d3196-a04c-40a0-9a08-f1cb72a4a870/volumes"
Oct 31 04:01:04.358019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.358004    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" path="/var/lib/kubelet/pods/b96ac7cb-59ab-4824-9c9f-6439c5d9b113/volumes"
Oct 31 04:01:04.358460 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:04.358447    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" path="/var/lib/kubelet/pods/e41e8673-22e9-4fd9-b3b2-43154730fc18/volumes"
Oct 31 04:01:06.382551 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:06.382516    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:01:07.664624 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:07.664586    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:08.987229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:08.987170    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:14.259529 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:14.259470    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:14.429137 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:14.429104    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:14.901315 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:14.901278    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:15.250796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.250761    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:15.468112 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468076    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"]
Oct 31 04:01:15.468864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468835    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy"
Oct 31 04:01:15.468864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468853    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468880    5906 cpu_manager.go:414] "containerMap: removing container" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468886    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468899    5906 cpu_manager.go:414] "containerMap: removing container" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468906    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468923    5906 cpu_manager.go:414] "containerMap: removing container" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468928    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468949    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468955    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468966    5906 cpu_manager.go:414] "containerMap: removing container" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468972    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468979    5906 cpu_manager.go:414] "containerMap: removing container" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468986    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.468995    5906 cpu_manager.go:414] "containerMap: removing container" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469001    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469020    5906 cpu_manager.go:414] "containerMap: removing container" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy"
Oct 31 04:01:15.469025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469028    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469172    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fb16942-2466-4501-8b09-eb90508d79c2" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469195    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e707f40-eb62-40b1-a9da-f302f66c83ee" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469228    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b96ac7cb-59ab-4824-9c9f-6439c5d9b113" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469240    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="e41e8673-22e9-4fd9-b3b2-43154730fc18" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469263    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469271    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="00104874-4cc9-4129-8923-0e301106d24e" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469282    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="82e25961-bd59-441a-9b0a-855dee1f6f47" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469294    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a79d3196-a04c-40a0-9a08-f1cb72a4a870" containerName="istio-proxy"
Oct 31 04:01:15.469583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.469301    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="234cd54a-a721-4edb-834a-01d4a9c72042" containerName="istio-proxy"
Oct 31 04:01:15.476590 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.476568    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.570502 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.559624    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-f9a7fa-openshift-default-dockercfg-trptc\""
Oct 31 04:01:15.594247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.588840    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631123    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631224    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631273    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631357    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631520    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631562    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631610    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631676    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.637230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.631733    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzwnt\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776264    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776338    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776377    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776452    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776594    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776633    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776694    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776757    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.776810    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzwnt\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.784235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.777985    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.837180 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.819730    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.865263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.837659    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.865263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.837901    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:15.969232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:15.943648    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:16.019565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.007435    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:16.043639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.043600    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"]
Oct 31 04:01:16.065758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.047471    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:16.096282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.093054    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:16.117241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.110125    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:16.117241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.115891    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzwnt\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt\") pod \"e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") " pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:16.393280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.392559    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:16.901073 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.901038    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"]
Oct 31 04:01:16.917797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.917755    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:16.917928 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.917849    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:16.917928 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:16.917896    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:17.877607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:17.876378    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" event={"ID":"e2246c16-a828-4f23-9615-06911991656e","Type":"ContainerStarted","Data":"14f8cd699153e2d0c3c5dcebd0bc82aceac5d310faf73dc9c2f2225912e19313"}
Oct 31 04:01:17.877607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:17.876460    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" event={"ID":"e2246c16-a828-4f23-9615-06911991656e","Type":"ContainerStarted","Data":"0efeef1292615fc8cc2641fd2b98ca8fd8fcec38292fd120a8cc025dee468609"}
Oct 31 04:01:18.398957 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.398915    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:18.416644 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.416607    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:18.549835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.549767    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podStartSLOduration=3.549746901 podStartE2EDuration="3.549746901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:18.077588531 +0000 UTC m=+1927.911448847" watchObservedRunningTime="2026-10-31 04:01:18.549746901 +0000 UTC m=+1928.383607206"
Oct 31 04:01:18.918258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.914390    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:18.918258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.917586    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:18.918258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.917835    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:18.938237 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:18.925474    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:01:19.417616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:19.417558    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:19.522278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:19.517864    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:19.927950 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:19.914613    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:19.927950 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:19.915437    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 04:01:19.951136 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:19.950251    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:19.989479 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:19.989448    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 04:01:20.227981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:20.220299    5906 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:20.874760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:20.874722    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:21.040961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:21.040927    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:21.184992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:21.184903    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:24.054548 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.054508    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:24.238471 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.238381    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:24.422297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.422130    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:24.496533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.496491    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:01:24.496726 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.496554    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:01:24.503525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.503488    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"]
Oct 31 04:01:24.512196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.512158    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.562119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.562069    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"]
Oct 31 04:01:24.674282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.611701    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrftn\" (UniqueName: \"kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn\") pod \"payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.674282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.611974    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.674282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.612455    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-f9a7fa-dockercfg-tngv9\""
Oct 31 04:01:24.674282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.613091    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-f9a7fa\""
Oct 31 04:01:24.717290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.717050    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrftn\" (UniqueName: \"kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn\") pod \"payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.718998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.717614    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.718998 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.718613    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.758578 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.758544    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"]
Oct 31 04:01:24.765084 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.765049    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"]
Oct 31 04:01:24.765280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.765166    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:24.782797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.782765    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrftn\" (UniqueName: \"kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn\") pod \"payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.831649 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.831612    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:24.921386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.920662    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:24.921386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.920813    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-627tb\" (UniqueName: \"kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb\") pod \"payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:24.928602 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:24.928495    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:25.022135 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.022098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:25.022381 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.022223    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-627tb\" (UniqueName: \"kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb\") pod \"payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:25.022917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.022895    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:25.085659 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.085617    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-627tb\" (UniqueName: \"kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb\") pod \"payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:25.105657 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.105621    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:25.258595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.258557    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"]
Oct 31 04:01:25.485486 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:01:25.485453    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4c0fc0_96a8_4bda_847c_9652c1c44aa1.slice/crio-1adee5db877b37d80c6b8249f04b6d7868388ad194e2780ddb3c28a0dd86c69e WatchSource:0}: Error finding container 1adee5db877b37d80c6b8249f04b6d7868388ad194e2780ddb3c28a0dd86c69e: Status 404 returned error can't find the container with id 1adee5db877b37d80c6b8249f04b6d7868388ad194e2780ddb3c28a0dd86c69e
Oct 31 04:01:25.489158 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.489065    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"]
Oct 31 04:01:25.741606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:25.741572    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:26.010042 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.010008    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7" event={"ID":"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1","Type":"ContainerStarted","Data":"3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef"}
Oct 31 04:01:26.010245 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.010069    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7" event={"ID":"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1","Type":"ContainerStarted","Data":"1adee5db877b37d80c6b8249f04b6d7868388ad194e2780ddb3c28a0dd86c69e"}
Oct 31 04:01:26.010312 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.010244    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:26.011657 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.011633    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn" event={"ID":"32753339-e8f8-4629-88da-e6ba8ffb6180","Type":"ContainerStarted","Data":"b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71"}
Oct 31 04:01:26.011789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.011674    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn" event={"ID":"32753339-e8f8-4629-88da-e6ba8ffb6180","Type":"ContainerStarted","Data":"f40e0eb7800109a29364c8e97b3a277906df0c25a0b50323750e39ba15d15f1d"}
Oct 31 04:01:26.011789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.011729    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:26.013266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.013245    5906 generic.go:356] "Generic (PLEG): container finished" podID="22ae802c-e968-4325-9415-0ff3be0e670b" containerID="f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513" exitCode=0
Oct 31 04:01:26.013351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.013312    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" event={"ID":"22ae802c-e968-4325-9415-0ff3be0e670b","Type":"ContainerDied","Data":"f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"}
Oct 31 04:01:26.013653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.013640    5906 scope.go:122] "RemoveContainer" containerID="f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"
Oct 31 04:01:26.015180 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.015155    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:26.015266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.015239    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:26.015306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.015273    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:26.144917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.144810    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7" podStartSLOduration=2.1447935830000002 podStartE2EDuration="2.144793583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:26.141352026 +0000 UTC m=+1935.975212332" watchObservedRunningTime="2026-10-31 04:01:26.144793583 +0000 UTC m=+1935.978653888"
Oct 31 04:01:26.306680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.306577    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn" podStartSLOduration=2.306562898 podStartE2EDuration="2.306562898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:26.305200631 +0000 UTC m=+1936.139060949" watchObservedRunningTime="2026-10-31 04:01:26.306562898 +0000 UTC m=+1936.140423202"
Oct 31 04:01:26.533236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.528269    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"]
Oct 31 04:01:26.704277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.685886    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:26.730232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.709086    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.812009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.711390    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"]
Oct 31 04:01:26.815582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815546    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.815890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815629    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.815890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815666    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.815890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815804    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.815890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815839    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.815890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815871    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.817322 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815927    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.817322 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.815959    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hszjn\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.817322 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.816033    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.872693 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.872538    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-833977-openshift-default-dockercfg-8bwxn\""
Oct 31 04:01:26.917027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.916979    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917029    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917154    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917284    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917328    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hszjn\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917383    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917416    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917468    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917508    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917589    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917630    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.917812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917723    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.918011 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.917985    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.918095 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.918045    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.919665 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.919640    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.920098 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.920076    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.981164 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.981077    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:26.981322 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:26.981200    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hszjn\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn\") pod \"e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") " pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:27.022835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.022802    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" event={"ID":"22ae802c-e968-4325-9415-0ff3be0e670b","Type":"ContainerStarted","Data":"d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65"}
Oct 31 04:01:27.092237 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.090336    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:27.092237 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.091019    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:27.594659 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:01:27.593738    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2a7874_12f4_49f8_8460_2932d97bf620.slice/crio-c7e83ed4ea6d8b8ecf263e4494a34736ede618a170fa82764421036de2aa2a75 WatchSource:0}: Error finding container c7e83ed4ea6d8b8ecf263e4494a34736ede618a170fa82764421036de2aa2a75: Status 404 returned error can't find the container with id c7e83ed4ea6d8b8ecf263e4494a34736ede618a170fa82764421036de2aa2a75
Oct 31 04:01:27.596489 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.596460    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"]
Oct 31 04:01:27.597100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.597072    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:27.597145 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.597137    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:27.597179 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.597165    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:27.934455 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.934372    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 04:01:27.934649 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.934620    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 04:01:27.934708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:27.934685    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 04:01:28.031411 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.031370    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" event={"ID":"4b2a7874-12f4-49f8-8460-2932d97bf620","Type":"ContainerStarted","Data":"9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"}
Oct 31 04:01:28.031604 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.031436    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" event={"ID":"4b2a7874-12f4-49f8-8460-2932d97bf620","Type":"ContainerStarted","Data":"c7e83ed4ea6d8b8ecf263e4494a34736ede618a170fa82764421036de2aa2a75"}
Oct 31 04:01:28.092069 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.092039    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:28.093409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.093390    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 31 04:01:28.093481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.093434    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 31 04:01:28.171226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.171034    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podStartSLOduration=2.171013369 podStartE2EDuration="2.171013369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:28.166286116 +0000 UTC m=+1938.000146422" watchObservedRunningTime="2026-10-31 04:01:28.171013369 +0000 UTC m=+1938.004873676"
Oct 31 04:01:28.934608 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.934568    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 04:01:28.935043 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:28.934624    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 04:01:29.096323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:29.096292    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:29.393794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:29.393751    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:29.934326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:29.934285    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 04:01:29.934512 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:29.934345    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 04:01:30.067738 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:30.067698    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:30.069015 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:30.068988    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:01:30.934792 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:30.934755    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 04:01:30.935003 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:30.934820    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 04:01:30.939502 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:30.939454    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:31.154230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:31.145927    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:31.342643 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:31.342583    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:31.934541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:31.934500    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused" start-of-body=
Oct 31 04:01:31.934749 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:31.934564    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.106:15021/healthz/ready\": dial tcp 10.134.0.106:15021: connect: connection refused"
Oct 31 04:01:32.938888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:32.938809    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 04:01:32.939425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:32.939193    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 04:01:32.939933 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:32.939913    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 04:01:34.094805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.094773    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"]
Oct 31 04:01:34.099331 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.099311    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.156999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.156958    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"]
Oct 31 04:01:34.163807 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.163757    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.163986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.163951    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.164072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.164034    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.164139 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.164095    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crjbm\" (UniqueName: \"kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.203036 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.203006    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-5a5baa-serving-cert\""
Oct 31 04:01:34.265720 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.265674    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.265937 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.265745    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.265937 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.265790    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crjbm\" (UniqueName: \"kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.266084 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.266066    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.266268 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:34.266247    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-serving-cert: secret "maas-api-e2e-shared-a-w0-5a5baa-serving-cert" not found
Oct 31 04:01:34.266352 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:34.266311    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls podName:02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e nodeName:}" failed. No retries permitted until 2026-10-31 04:01:34.766295918 +0000 UTC m=+1944.600156213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls") pod "maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" (UID: "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e") : secret "maas-api-e2e-shared-a-w0-5a5baa-serving-cert" not found
Oct 31 04:01:34.266417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.266382    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.268425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.268401    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.269827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.269788    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:34.332320 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.332281    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crjbm\" (UniqueName: \"kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.472355 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.472235    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:34.705901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.705861    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:34.792783 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.791397    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.795807 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.795780    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:34.932230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:34.927297    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:35.013738 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.013662    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:35.057171 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.057084    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"]
Oct 31 04:01:35.062376 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.062350    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.104794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.104763    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xftd8\" (UniqueName: \"kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8\") pod \"payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.105261 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.105182    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.121822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.121784    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"]
Oct 31 04:01:35.125433 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.125406    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-833977\""
Oct 31 04:01:35.127137 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.127111    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"]
Oct 31 04:01:35.149228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.134776    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.242229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.238724    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-833977-dockercfg-h6mgv\""
Oct 31 04:01:35.266232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.245292    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9flgq\" (UniqueName: \"kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq\") pod \"payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.339682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.263121    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"]
Oct 31 04:01:35.339682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.314824    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.339682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.314883    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftd8\" (UniqueName: \"kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8\") pod \"payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.339682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.314927    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.339682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.316101    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.379550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.379517    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftd8\" (UniqueName: \"kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8\") pod \"payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.380014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.379985    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:35.416621 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.416588    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.416763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.416744    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9flgq\" (UniqueName: \"kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq\") pod \"payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.417406 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.417385    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.483475 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.483444    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9flgq\" (UniqueName: \"kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq\") pod \"payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.535711 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.535684    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:01:35.536157 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.536127    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"]
Oct 31 04:01:35.675055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.674970    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:35.788600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:35.788568    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"]
Oct 31 04:01:36.049536 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:01:36.049498    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68df62e_2efc_4571_8f12_f68b978d8407.slice/crio-78dd27e5a73ff1617402f3ab24791d26651549d1473c9b64d7ee6fc7c1335cf0 WatchSource:0}: Error finding container 78dd27e5a73ff1617402f3ab24791d26651549d1473c9b64d7ee6fc7c1335cf0: Status 404 returned error can't find the container with id 78dd27e5a73ff1617402f3ab24791d26651549d1473c9b64d7ee6fc7c1335cf0
Oct 31 04:01:36.052696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.052665    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"]
Oct 31 04:01:36.120433 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.120401    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl" event={"ID":"83dec427-47e3-49f6-8cf3-134b41131e40","Type":"ContainerStarted","Data":"f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725"}
Oct 31 04:01:36.120835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.120456    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl" event={"ID":"83dec427-47e3-49f6-8cf3-134b41131e40","Type":"ContainerStarted","Data":"77e3363eb3f00bea22cc85f300749d3ec1c3c931a8a4f959a504fd4353b6bc37"}
Oct 31 04:01:36.120835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.120623    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:36.122399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.122373    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m" event={"ID":"c68df62e-2efc-4571-8f12-f68b978d8407","Type":"ContainerStarted","Data":"78dd27e5a73ff1617402f3ab24791d26651549d1473c9b64d7ee6fc7c1335cf0"}
Oct 31 04:01:36.123847 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.123822    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" event={"ID":"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e","Type":"ContainerStarted","Data":"f93ae3a69110a614a518b8b782961ade87fcbc69e3a19a3b6367e989666edabd"}
Oct 31 04:01:36.259003 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:36.258949    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl" podStartSLOduration=1.258934323 podStartE2EDuration="1.258934323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:36.258314121 +0000 UTC m=+1946.092174496" watchObservedRunningTime="2026-10-31 04:01:36.258934323 +0000 UTC m=+1946.092794637"
Oct 31 04:01:36.435292 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:01:36.435185    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:01:37.026077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.026046    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:01:37.026267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.026119    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:01:37.139303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.139262    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" event={"ID":"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e","Type":"ContainerStarted","Data":"8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c"}
Oct 31 04:01:37.145361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.145325    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:37.162763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.161804    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m" event={"ID":"c68df62e-2efc-4571-8f12-f68b978d8407","Type":"ContainerStarted","Data":"590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30"}
Oct 31 04:01:37.162763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.161893    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:37.454691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.454577    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m" podStartSLOduration=2.454555052 podStartE2EDuration="2.454555052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:37.453683471 +0000 UTC m=+1947.287543777" watchObservedRunningTime="2026-10-31 04:01:37.454555052 +0000 UTC m=+1947.288415360"
Oct 31 04:01:37.591702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:37.591634    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" podStartSLOduration=2.575798738 podStartE2EDuration="3.591616354s" totalImagesPullingTime="1.015817616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:37.578782558 +0000 UTC m=+1947.412642864" watchObservedRunningTime="2026-10-31 04:01:37.591616354 +0000 UTC m=+1947.425476704"
Oct 31 04:01:38.385610 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:38.385580    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:39.281029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:39.280995    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:40.193276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.193243    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:40.743189 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.743137    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"]
Oct 31 04:01:40.749046 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.749011    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.806313 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.806279    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ada9ed-openshift-default-dockercfg-7rlkw\""
Oct 31 04:01:40.825238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.825182    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.826466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.826437    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.826701 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.826681    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.826862 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.826847    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.827017 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.827005    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.838262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.828374    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.838262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.828436    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds58d\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.838262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.828516    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.838262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.828563    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.939062 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.939004    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.939400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.939376    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.939639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.939619    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.939844 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.939825    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.939987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.939972    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.940185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.940159    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.940422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.940391    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.940620 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.940592    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.940811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.940782    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds58d\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.941883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.941840    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.957300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.947584    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.957300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.948123    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:40.957300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.955984    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.970828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.970776    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:40.974963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:40.972570    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:41.006309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.000586    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"]
Oct 31 04:01:41.032080 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.032045    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:41.036298 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.035459    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:41.053600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.038702    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:41.153227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.146768    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds58d\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d\") pod \"e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") " pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:41.322784 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.321437    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:41.363881 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.363846    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:41.789085 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:01:41.789055    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2691af5_1c21_407e_9627_0ebc83d05b86.slice/crio-843b99f2aa8af1a9beef0684b6858559faf51056d7ec74af50ed76da7f6cc95b WatchSource:0}: Error finding container 843b99f2aa8af1a9beef0684b6858559faf51056d7ec74af50ed76da7f6cc95b: Status 404 returned error can't find the container with id 843b99f2aa8af1a9beef0684b6858559faf51056d7ec74af50ed76da7f6cc95b
Oct 31 04:01:41.791878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.791842    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:41.792005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.791909    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:41.792005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.791909    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"]
Oct 31 04:01:41.792005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:41.791938    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:01:42.204462 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:42.204370    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" event={"ID":"d2691af5-1c21-407e-9627-0ebc83d05b86","Type":"ContainerStarted","Data":"d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"}
Oct 31 04:01:42.204462 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:42.204434    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" event={"ID":"d2691af5-1c21-407e-9627-0ebc83d05b86","Type":"ContainerStarted","Data":"843b99f2aa8af1a9beef0684b6858559faf51056d7ec74af50ed76da7f6cc95b"}
Oct 31 04:01:42.330067 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:42.330011    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podStartSLOduration=2.32999405 podStartE2EDuration="2.32999405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:42.327683309 +0000 UTC m=+1952.161543615" watchObservedRunningTime="2026-10-31 04:01:42.32999405 +0000 UTC m=+1952.163854357"
Oct 31 04:01:42.364277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:42.364244    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:42.370441 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:42.370413    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:43.210981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:43.210945    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:43.211985 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:43.211963    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:01:43.793357 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:43.793323    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:44.182722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:44.182643    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:01:45.103767 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:45.103724    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:45.267097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:45.267054    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:45.418621 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:45.418540    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:47.163716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:47.163688    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:01:48.171109 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.171079    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:01:48.258183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.258148    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:48.414756 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.414715    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:48.678024 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.677992    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"]
Oct 31 04:01:48.683757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.683687    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.749256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.742974    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-ada9ed\""
Oct 31 04:01:48.749256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.743181    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-ada9ed-dockercfg-9f7jt\""
Oct 31 04:01:48.771679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.771630    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.778234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.772236    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcwkf\" (UniqueName: \"kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf\") pod \"payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.901486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.901424    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.903450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.903416    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcwkf\" (UniqueName: \"kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf\") pod \"payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.904099 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.902686    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.916922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.916887    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"]
Oct 31 04:01:48.921750 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.921721    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"]
Oct 31 04:01:48.930398 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.930319    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:48.963468 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.957963    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"]
Oct 31 04:01:48.970082 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.970046    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcwkf\" (UniqueName: \"kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf\") pod \"payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:48.980992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:48.980863    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:49.009531 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.009477    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:49.112253 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.109347    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npz2v\" (UniqueName: \"kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v\") pod \"payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") " pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.112253 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.109536    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") " pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.211380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.211284    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") " pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.211773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.211592    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npz2v\" (UniqueName: \"kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v\") pod \"payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") " pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.212262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.212235    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume\") pod \"payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") " pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.274461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.274432    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npz2v\" (UniqueName: \"kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v\") pod \"payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") " pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.296166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.296134    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:49.468197 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:01:49.468164    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1828d0f0_cdbc_4f98_a047_a6ecb71d0a69.slice/crio-83dbcae9558b6d70cec3201af71011984c8e349903b4b87e4bd831c63a9bad88 WatchSource:0}: Error finding container 83dbcae9558b6d70cec3201af71011984c8e349903b4b87e4bd831c63a9bad88: Status 404 returned error can't find the container with id 83dbcae9558b6d70cec3201af71011984c8e349903b4b87e4bd831c63a9bad88
Oct 31 04:01:49.469063 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.469040    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"]
Oct 31 04:01:49.682808 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:01:49.682772    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48c9ffad_9df1_429a_b483_170065a30abb.slice/crio-26f553e6b146337688d5f703fcdf866bd31dad1679e1744eb6cb44ab9808adae WatchSource:0}: Error finding container 26f553e6b146337688d5f703fcdf866bd31dad1679e1744eb6cb44ab9808adae: Status 404 returned error can't find the container with id 26f553e6b146337688d5f703fcdf866bd31dad1679e1744eb6cb44ab9808adae
Oct 31 04:01:49.686413 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:49.686385    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"]
Oct 31 04:01:50.260219 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.260165    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt" event={"ID":"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69","Type":"ContainerStarted","Data":"83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3"}
Oct 31 04:01:50.261019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.260274    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt" event={"ID":"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69","Type":"ContainerStarted","Data":"83dbcae9558b6d70cec3201af71011984c8e349903b4b87e4bd831c63a9bad88"}
Oct 31 04:01:50.261019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.260389    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:01:50.261986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.261958    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr" event={"ID":"48c9ffad-9df1-429a-b483-170065a30abb","Type":"ContainerStarted","Data":"879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e"}
Oct 31 04:01:50.262121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.262006    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr" event={"ID":"48c9ffad-9df1-429a-b483-170065a30abb","Type":"ContainerStarted","Data":"26f553e6b146337688d5f703fcdf866bd31dad1679e1744eb6cb44ab9808adae"}
Oct 31 04:01:50.262121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.262054    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:01:50.381505 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:50.381442    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt" podStartSLOduration=2.381423082 podStartE2EDuration="2.381423082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:50.378620645 +0000 UTC m=+1960.212480949" watchObservedRunningTime="2026-10-31 04:01:50.381423082 +0000 UTC m=+1960.215283391"
Oct 31 04:01:53.517641 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:53.517609    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:01:58.011780 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:01:58.011741    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:01.271856 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:01.271821    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:02:01.272407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:01.271897    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:02:01.396404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:01.396344    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr" podStartSLOduration=13.396330724 podStartE2EDuration="13.396330724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:01:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:01:50.558721721 +0000 UTC m=+1960.392582029" watchObservedRunningTime="2026-10-31 04:02:01.396330724 +0000 UTC m=+1971.230191069"
Oct 31 04:02:02.494682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:02.494650    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:03.354894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:03.354801    5906 generic.go:356] "Generic (PLEG): container finished" podID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerID="08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c" exitCode=0
Oct 31 04:02:03.354894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:03.354870    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" event={"ID":"c44d273e-2eb6-41a7-a91e-35905e76ab88","Type":"ContainerDied","Data":"08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"}
Oct 31 04:02:03.355386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:03.355371    5906 scope.go:122] "RemoveContainer" containerID="08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"
Oct 31 04:02:03.357171 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:03.357135    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:02:03.357326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:03.357244    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:02:03.357326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:03.357298    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:02:04.370770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:04.370721    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" event={"ID":"c44d273e-2eb6-41a7-a91e-35905e76ab88","Type":"ContainerStarted","Data":"c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76"}
Oct 31 04:02:04.436609 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:04.436572    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:02:04.436832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:04.436763    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 04:02:04.436925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:04.436824    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 04:02:05.436823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:05.436783    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 04:02:05.437253 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:05.436850    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 04:02:06.392746 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:06.392698    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:02:06.437119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:06.437081    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 04:02:06.437531 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:06.437146    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 04:02:06.911447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:06.911417    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:07.436500 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:07.436459    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 04:02:07.436721 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:07.436528    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 04:02:08.436783 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:08.436743    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 04:02:08.437191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:08.436797    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 04:02:08.881423 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:08.881377    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:09.022563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:09.022525    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:09.437327 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:09.437288    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 31 04:02:09.437737 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:09.437344    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 31 04:02:10.441738 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:10.441708    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:02:10.442256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:10.442117    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:02:10.442960 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:10.442938    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:02:11.383817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:11.383778    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:15.878256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:15.871384    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:20.308396 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:20.308303    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:24.571234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.571156    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"]
Oct 31 04:02:24.577446 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.577416    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.631908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.631866    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"]
Oct 31 04:02:24.644791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.644752    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkdq5\" (UniqueName: \"kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.644977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.644894    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.645046 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.644991    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.645136 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.645113    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.678774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.678739    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-f9a7fa-serving-cert\""
Oct 31 04:02:24.746805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.746770    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkdq5\" (UniqueName: \"kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.747019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.746868    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.747019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.746992    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.747157 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.747045    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.747157 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:24.747064    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-serving-cert: secret "maas-api-e2e-shared-b-w1-f9a7fa-serving-cert" not found
Oct 31 04:02:24.747157 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:24.747130    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls podName:9e2ce747-c974-460c-b44d-c4e6f0c62734 nodeName:}" failed. No retries permitted until 2026-10-31 04:02:25.247110163 +0000 UTC m=+1995.080970462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls") pod "maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" (UID: "9e2ce747-c974-460c-b44d-c4e6f0c62734") : secret "maas-api-e2e-shared-b-w1-f9a7fa-serving-cert" not found
Oct 31 04:02:24.747774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.747742    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.748638 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.748612    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:24.749785 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.749763    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.831317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.831217    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkdq5\" (UniqueName: \"kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:24.905978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:24.905923    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:25.052805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:25.052772    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:25.274184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:25.274141    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:25.276963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:25.276935    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:25.277977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:25.277952    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:25.494292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:25.494257    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:25.873364 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:02:25.873325    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2ce747_c974_460c_b44d_c4e6f0c62734.slice/crio-f2c5c1b23dd441d741ffe613e121cfc7c7b463b602da6a3b113bc145f10ac933 WatchSource:0}: Error finding container f2c5c1b23dd441d741ffe613e121cfc7c7b463b602da6a3b113bc145f10ac933: Status 404 returned error can't find the container with id f2c5c1b23dd441d741ffe613e121cfc7c7b463b602da6a3b113bc145f10ac933
Oct 31 04:02:25.877379 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:25.877345    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"]
Oct 31 04:02:26.526374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:26.526329    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" event={"ID":"9e2ce747-c974-460c-b44d-c4e6f0c62734","Type":"ContainerStarted","Data":"f2c5c1b23dd441d741ffe613e121cfc7c7b463b602da6a3b113bc145f10ac933"}
Oct 31 04:02:27.539645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:27.539601    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" event={"ID":"9e2ce747-c974-460c-b44d-c4e6f0c62734","Type":"ContainerStarted","Data":"aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755"}
Oct 31 04:02:27.540161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:27.540137    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:27.664174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:27.664114    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" podStartSLOduration=2.7546967540000002 podStartE2EDuration="3.66409752s" totalImagesPullingTime="909.400766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:02:27.660510104 +0000 UTC m=+1997.494370412" watchObservedRunningTime="2026-10-31 04:02:27.66409752 +0000 UTC m=+1997.497957825"
Oct 31 04:02:29.425983 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:29.425902    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:32.608284 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:32.608239    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:32.770166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:32.770137    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:33.552918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:33.552886    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:02:34.009488 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:34.009456    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:36.383870 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:36.383831    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:02:38.560497 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.560466    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"]
Oct 31 04:02:38.565310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.565282    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.598817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.598782    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4r8\" (UniqueName: \"kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.598999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.598847    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.599079 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.599042    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.599144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.599103    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.619303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.619275    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-833977-serving-cert\""
Oct 31 04:02:38.676709 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.676661    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"]
Oct 31 04:02:38.700659 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.700603    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4r8\" (UniqueName: \"kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.700867 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.700702    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.700867 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.700794    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.700867 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.700829    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.701174 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:38.701147    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-serving-cert: secret "maas-api-e2e-shared-b-w3-833977-serving-cert" not found
Oct 31 04:02:38.701367 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:38.701271    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls podName:20f715b5-f7d0-440c-9401-dc9cf82944e9 nodeName:}" failed. No retries permitted until 2026-10-31 04:02:39.201245406 +0000 UTC m=+2009.035105709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls") pod "maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" (UID: "20f715b5-f7d0-440c-9401-dc9cf82944e9") : secret "maas-api-e2e-shared-b-w3-833977-serving-cert" not found
Oct 31 04:02:38.701848 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.701823    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.703970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.703946    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.764473 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.764440    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4r8\" (UniqueName: \"kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:38.855712 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:38.855624    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:39.003226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:39.000761    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:39.220447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:39.220334    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:39.223121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:39.223089    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:39.481543 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:39.481440    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:39.859105 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:02:39.859074    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f715b5_f7d0_440c_9401_dc9cf82944e9.slice/crio-14538c63440a3501c311a5288e0ebda134cf4d0895e13b715e1f4df711f766ac WatchSource:0}: Error finding container 14538c63440a3501c311a5288e0ebda134cf4d0895e13b715e1f4df711f766ac: Status 404 returned error can't find the container with id 14538c63440a3501c311a5288e0ebda134cf4d0895e13b715e1f4df711f766ac
Oct 31 04:02:39.862593 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:39.862565    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"]
Oct 31 04:02:40.675509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:40.675466    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" event={"ID":"20f715b5-f7d0-440c-9401-dc9cf82944e9","Type":"ContainerStarted","Data":"14538c63440a3501c311a5288e0ebda134cf4d0895e13b715e1f4df711f766ac"}
Oct 31 04:02:41.684430 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:41.684391    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" event={"ID":"20f715b5-f7d0-440c-9401-dc9cf82944e9","Type":"ContainerStarted","Data":"603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b"}
Oct 31 04:02:41.684852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:41.684493    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:41.807354 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:41.807294    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" podStartSLOduration=2.797124434 podStartE2EDuration="3.807279215s" totalImagesPullingTime="1.010154781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:02:41.805494745 +0000 UTC m=+2011.639355051" watchObservedRunningTime="2026-10-31 04:02:41.807279215 +0000 UTC m=+2011.641139519"
Oct 31 04:02:43.492942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:43.492902    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:46.071233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:46.070927    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:46.210916 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:46.210874    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:46.863234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:46.863178    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:47.720640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:47.720606    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:02:48.069948 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:48.069912    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:48.731802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:48.731770    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:52.758908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:52.758871    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:53.452982 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.452886    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:53.786052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.786019    5906 generic.go:356] "Generic (PLEG): container finished" podID="e2246c16-a828-4f23-9615-06911991656e" containerID="14f8cd699153e2d0c3c5dcebd0bc82aceac5d310faf73dc9c2f2225912e19313" exitCode=0
Oct 31 04:02:53.786588 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.786098    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" event={"ID":"e2246c16-a828-4f23-9615-06911991656e","Type":"ContainerDied","Data":"14f8cd699153e2d0c3c5dcebd0bc82aceac5d310faf73dc9c2f2225912e19313"}
Oct 31 04:02:53.786686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.786667    5906 scope.go:122] "RemoveContainer" containerID="14f8cd699153e2d0c3c5dcebd0bc82aceac5d310faf73dc9c2f2225912e19313"
Oct 31 04:02:53.788351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.788322    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:02:53.788479 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.788406    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:02:53.788479 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:53.788458    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:02:54.795981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:54.795943    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" event={"ID":"e2246c16-a828-4f23-9615-06911991656e","Type":"ContainerStarted","Data":"703014f2811c89030c1944d1070c7d8f67e2294891d815fa996344efd7b4ba64"}
Oct 31 04:02:55.392842 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:55.392801    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:02:55.393066 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:55.393004    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 31 04:02:55.393148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:55.393073    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 31 04:02:56.393200 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:56.393163    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 31 04:02:56.393635 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:56.393248    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 31 04:02:57.394179 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.394128    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 31 04:02:57.394732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.394377    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 31 04:02:57.398128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.398093    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"]
Oct 31 04:02:57.403100 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.403075    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.416074 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.416046    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq4np\" (UniqueName: \"kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.416177 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.416097    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.416259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.416193    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.416346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.416333    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.470570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.470532    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"]
Oct 31 04:02:57.514973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.514939    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ada9ed-serving-cert\""
Oct 31 04:02:57.517865 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.517833    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq4np\" (UniqueName: \"kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.517992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.517887    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.517992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.517940    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.518087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.518013    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.518169 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:57.518155    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-serving-cert: secret "maas-api-e2e-api-a-ada9ed-serving-cert" not found
Oct 31 04:02:57.518251 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:02:57.518223    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls podName:a79b0003-c031-4b13-9375-97a1d0ccefbb nodeName:}" failed. No retries permitted until 2026-10-31 04:02:58.018191345 +0000 UTC m=+2027.852051642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls") pod "maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" (UID: "a79b0003-c031-4b13-9375-97a1d0ccefbb") : secret "maas-api-e2e-api-a-ada9ed-serving-cert" not found
Oct 31 04:02:57.519822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.519797    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.522669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.522645    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.577003 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.576971    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:57.582916 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.582698    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq4np\" (UniqueName: \"kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:57.786150 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.786112    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:57.934360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:57.934316    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:58.075445 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.075359    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:58.077940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.077917    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\") pod \"maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:58.319069 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.319021    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:02:58.376677 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.376647    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:58.394519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.394224    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 31 04:02:58.394519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.394280    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 31 04:02:58.734978 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:02:58.734929    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79b0003_c031_4b13_9375_97a1d0ccefbb.slice/crio-f836c3c405ce1e4e059be2c9ef258eb2b0cb3480b716a4334d214ed9801b58b2 WatchSource:0}: Error finding container f836c3c405ce1e4e059be2c9ef258eb2b0cb3480b716a4334d214ed9801b58b2: Status 404 returned error can't find the container with id f836c3c405ce1e4e059be2c9ef258eb2b0cb3480b716a4334d214ed9801b58b2
Oct 31 04:02:58.736018 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.735991    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"]
Oct 31 04:02:58.873242 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.873192    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:02:58.880004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:58.879969    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" event={"ID":"a79b0003-c031-4b13-9375-97a1d0ccefbb","Type":"ContainerStarted","Data":"f836c3c405ce1e4e059be2c9ef258eb2b0cb3480b716a4334d214ed9801b58b2"}
Oct 31 04:02:59.393850 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:59.393818    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 31 04:02:59.394035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:59.393899    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 31 04:02:59.889222 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:59.889172    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" event={"ID":"a79b0003-c031-4b13-9375-97a1d0ccefbb","Type":"ContainerStarted","Data":"1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118"}
Oct 31 04:02:59.889757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:02:59.889728    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:03:00.071620 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.071394    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" podStartSLOduration=2.14183271 podStartE2EDuration="3.071367292s" totalImagesPullingTime="929.534582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:03:00.062988209 +0000 UTC m=+2029.896848514" watchObservedRunningTime="2026-10-31 04:03:00.071367292 +0000 UTC m=+2029.905227598"
Oct 31 04:03:00.402126 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.402017    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:03:00.402534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.402507    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:03:00.403613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.403587    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:03:00.604279 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.592418    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"]
Oct 31 04:03:00.690833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.659085    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:00.690833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.676473    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.738805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.737712    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"]
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.806571    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-288b63-openshift-default-dockercfg-hgk7w\""
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829112    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829163    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829253    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829360    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829411    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829458    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwzgx\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829493    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829653    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.830878 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.829742    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.833866 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.833832    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:00.935578 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935545    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935615    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwzgx\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935656    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935781    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935808    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935873    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935905    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.935974    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.936039    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936508 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.936122    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.936931 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.936904    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.937177 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.937153    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.937380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.937272    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.937732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.937714    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.940727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.940701    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.940932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.940883    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:00.994920 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:00.994750    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:01.024128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.021068    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwzgx\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:01.036316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.024808    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token\") pod \"e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") " pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:01.136310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.136269    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:01.562724 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:03:01.562687    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod856a336a_5439_48b5_af95_01572728f411.slice/crio-4af2520bac72a89a849d2dae5fbb9d51ce1e496d3b176fae341345c48175958e WatchSource:0}: Error finding container 4af2520bac72a89a849d2dae5fbb9d51ce1e496d3b176fae341345c48175958e: Status 404 returned error can't find the container with id 4af2520bac72a89a849d2dae5fbb9d51ce1e496d3b176fae341345c48175958e
Oct 31 04:03:01.565776 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.565735    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:01.565858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.565829    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:01.565909 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.565873    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:01.566102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.566079    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"]
Oct 31 04:03:01.915576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.915475    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" event={"ID":"856a336a-5439-48b5-af95-01572728f411","Type":"ContainerStarted","Data":"afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"}
Oct 31 04:03:01.915576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:01.915547    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" event={"ID":"856a336a-5439-48b5-af95-01572728f411","Type":"ContainerStarted","Data":"4af2520bac72a89a849d2dae5fbb9d51ce1e496d3b176fae341345c48175958e"}
Oct 31 04:03:02.044716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:02.044652    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podStartSLOduration=2.044633427 podStartE2EDuration="2.044633427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:03:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:03:02.039448704 +0000 UTC m=+2031.873309008" watchObservedRunningTime="2026-10-31 04:03:02.044633427 +0000 UTC m=+2031.878493733"
Oct 31 04:03:02.136970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:02.136923    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:02.143396 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:02.143368    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:02.608540 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:02.608505    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:02.923193 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:02.923120    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:02.924340 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:02.924320    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:03:04.948036 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:04.947998    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:05.109879 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:05.109843    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:05.257291 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:05.257261    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:05.843429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:05.843395    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:05.911588 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:05.911556    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:03:06.384078 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:03:06.384041    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:03:07.449497 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:07.449464    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:07.757883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:07.757851    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:08.036194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:08.036100    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:08.251455 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:08.251415    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:12.573780 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:12.573747    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:17.315528 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:17.315475    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:18.776965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:18.776907    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:19.569374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:19.569300    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:21.325430 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.325375    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"]
Oct 31 04:03:21.326071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.325622    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq" podUID="e6e690ce-54a5-4d88-8a7e-bc648fe9b483" containerName="maas-api" containerID="cri-o://31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105" gracePeriod=30
Oct 31 04:03:21.411567 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.411525    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:21.690286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.690258    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 04:03:21.839310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.839256    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\") pod \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") "
Oct 31 04:03:21.840047 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.840022    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs\") pod \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") "
Oct 31 04:03:21.844473 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.844437    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca\") pod \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") "
Oct 31 04:03:21.844808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.844785    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc\" (UniqueName: \"kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc\") pod \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\" (UID: \"e6e690ce-54a5-4d88-8a7e-bc648fe9b483\") "
Oct 31 04:03:21.853276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.843562    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls" pod "e6e690ce-54a5-4d88-8a7e-bc648fe9b483" (UID: "e6e690ce-54a5-4d88-8a7e-bc648fe9b483"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:03:21.853276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.849601    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca" pod "e6e690ce-54a5-4d88-8a7e-bc648fe9b483" (UID: "e6e690ce-54a5-4d88-8a7e-bc648fe9b483"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:03:21.868985 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.859672    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc" pod "e6e690ce-54a5-4d88-8a7e-bc648fe9b483" (UID: "e6e690ce-54a5-4d88-8a7e-bc648fe9b483"). InnerVolumeSpecName "kube-api-access-lzvmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:21.868985 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.859793    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs" pod "e6e690ce-54a5-4d88-8a7e-bc648fe9b483" (UID: "e6e690ce-54a5-4d88-8a7e-bc648fe9b483"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:03:21.870279 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.870161    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:21.870524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.870497    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:21.870774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.870759    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzvmc\" (UniqueName: \"kubernetes.io/projected/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-kube-api-access-lzvmc\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:21.870896 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:21.870882    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6e690ce-54a5-4d88-8a7e-bc648fe9b483-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:22.069768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.069734    5906 generic.go:356] "Generic (PLEG): container finished" podID="e6e690ce-54a5-4d88-8a7e-bc648fe9b483" containerID="31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105" exitCode=0
Oct 31 04:03:22.069977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.069781    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"
Oct 31 04:03:22.069977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.069824    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq" event={"ID":"e6e690ce-54a5-4d88-8a7e-bc648fe9b483","Type":"ContainerDied","Data":"31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105"}
Oct 31 04:03:22.069977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.069883    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq" event={"ID":"e6e690ce-54a5-4d88-8a7e-bc648fe9b483","Type":"ContainerDied","Data":"ee7c608b6724248a3aeb4538fc7c165d919ad9fe2658a7e4182b749f5971f972"}
Oct 31 04:03:22.069977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.069911    5906 scope.go:122] "RemoveContainer" containerID="31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105"
Oct 31 04:03:22.081848 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.081819    5906 scope.go:122] "RemoveContainer" containerID="31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105"
Oct 31 04:03:22.082159 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:03:22.082139    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105\": container with ID starting with 31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105 not found: ID does not exist" containerID="31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105"
Oct 31 04:03:22.082253 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.082169    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105"} err="failed to get container status \"31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105\": rpc error: code = NotFound desc = could not find container \"31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105\": container with ID starting with 31798ef9ba807a9282a9413d545bf236b9afbc8d4fdb75eb1c6a2d0450100105 not found: ID does not exist"
Oct 31 04:03:22.241869 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.241826    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"]
Oct 31 04:03:22.246949 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.246917    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-22e20532-86f689d595-7pjvq"]
Oct 31 04:03:22.353676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:22.353601    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6e690ce-54a5-4d88-8a7e-bc648fe9b483" path="/var/lib/kubelet/pods/e6e690ce-54a5-4d88-8a7e-bc648fe9b483/volumes"
Oct 31 04:03:23.050964 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:23.050926    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:23.370731 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:23.370608    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:23.561530 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:23.561496    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:25.631570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:25.631529    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:28.057761 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:28.057726    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:28.203631 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:28.203594    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:28.404673 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:28.404583    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:28.551883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:28.551846    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:29.606389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:29.606354    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:29.762810 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:29.762761    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:29.976257 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:29.976143    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:30.536565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:30.536529    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:32.772840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:32.772805    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:33.294424 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:33.294391    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:34.021648 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.021619    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:34.242682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.242650    5906 generic.go:356] "Generic (PLEG): container finished" podID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerID="d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d" exitCode=0
Oct 31 04:03:34.242901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.242724    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" event={"ID":"d2691af5-1c21-407e-9627-0ebc83d05b86","Type":"ContainerDied","Data":"d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"}
Oct 31 04:03:34.243323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.243302    5906 scope.go:122] "RemoveContainer" containerID="d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"
Oct 31 04:03:34.244758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.244731    5906 generic.go:356] "Generic (PLEG): container finished" podID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerID="9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb" exitCode=0
Oct 31 04:03:34.244883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.244809    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" event={"ID":"4b2a7874-12f4-49f8-8460-2932d97bf620","Type":"ContainerDied","Data":"9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"}
Oct 31 04:03:34.245199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.245177    5906 scope.go:122] "RemoveContainer" containerID="9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"
Oct 31 04:03:34.245347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.245322    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:34.245408 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.245395    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:34.245461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.245432    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:34.247127 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.247099    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:34.247201 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.247179    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:34.247278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:34.247262    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:03:35.293228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:35.293017    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" event={"ID":"4b2a7874-12f4-49f8-8460-2932d97bf620","Type":"ContainerStarted","Data":"92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e"}
Oct 31 04:03:35.297772 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:35.297691    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" event={"ID":"d2691af5-1c21-407e-9627-0ebc83d05b86","Type":"ContainerStarted","Data":"92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889"}
Oct 31 04:03:35.364484 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:35.364435    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:03:35.364785 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:35.364752    5906 patch_prober.go:29] interesting pod/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 31 04:03:35.365009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:35.364981    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 31 04:03:36.092841 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:36.091935    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 31 04:03:36.092841 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:36.092014    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 31 04:03:36.092841 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:36.092071    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:03:36.391359 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:36.381074    5906 patch_prober.go:29] interesting pod/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 31 04:03:36.391359 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:36.381284    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 31 04:03:36.421218 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:03:36.421175    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:03:37.091807 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:37.091768    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 31 04:03:37.092123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:37.092094    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 31 04:03:37.367820 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:37.367734    5906 patch_prober.go:29] interesting pod/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 31 04:03:37.368081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:37.368051    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 31 04:03:37.862373 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:37.862335    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:38.092549 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:38.092515    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 31 04:03:38.092811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:38.092782    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 31 04:03:38.406460 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:38.406423    5906 patch_prober.go:29] interesting pod/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 31 04:03:38.406763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:38.406732    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 31 04:03:39.093232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:39.092665    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 31 04:03:39.093232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:39.092733    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 31 04:03:39.364744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:39.364641    5906 patch_prober.go:29] interesting pod/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 31 04:03:39.365023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:39.364991    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 31 04:03:40.092121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.091656    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 31 04:03:40.092121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.091721    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 31 04:03:40.372288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.371353    5906 patch_prober.go:29] interesting pod/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused" start-of-body=
Oct 31 04:03:40.372288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.371429    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.122:15021/healthz/ready\": dial tcp 10.134.0.122:15021: connect: connection refused"
Oct 31 04:03:40.485427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.485395    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:40.628766 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.628668    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:40.848809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:40.848774    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:41.115099 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.115063    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:03:41.116966 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.116941    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:03:41.130852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.130810    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:03:41.346231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.346180    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:41.368815 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.368741    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:03:41.369023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.369005    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:03:41.369594 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:41.369575    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:03:42.438372 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:42.438340    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:46.465747 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:46.465713    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:47.285968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:47.285928    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:47.638683 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:47.638598    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:48.645877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:48.645784    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:49.559542 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:49.559495    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:50.884230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:50.883316    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:03:50.884230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:50.883457    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:03:50.923360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:50.923317    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:03:50.924904 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:50.924865    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:03:52.446397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:52.446356    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:53.862176 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:53.862140    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"]
Oct 31 04:03:54.101794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:54.101762    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:54.240702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:54.240620    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:54.384628 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:54.384585    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:57.298349 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:57.298311    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:03:58.863305 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:03:58.863260    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerName="payload-pre-processing" containerID="cri-o://691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e" gracePeriod=25
Oct 31 04:04:01.239365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:01.239316    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:04:02.013636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:02.013595    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:03.785901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:03.785863    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:03.923016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:03.922982    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:04.704157 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:04.704121    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:06.384894 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:04:06.384853    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:04:06.929077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:06.929048    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:11.210004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:11.209970    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:04:11.808440 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:11.808411    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:16.623314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:16.623281    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:17.672905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:17.672866    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:18.579172 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.579140    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"]
Oct 31 04:04:18.579963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.579945    5906 cpu_manager.go:414] "containerMap: removing container" podUID="e6e690ce-54a5-4d88-8a7e-bc648fe9b483" containerName="maas-api"
Oct 31 04:04:18.580009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.579966    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6e690ce-54a5-4d88-8a7e-bc648fe9b483" containerName="maas-api"
Oct 31 04:04:18.580072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.580063    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6e690ce-54a5-4d88-8a7e-bc648fe9b483" containerName="maas-api"
Oct 31 04:04:18.583652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.583633    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.645958 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.645923    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"]
Oct 31 04:04:18.676556 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.676514    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:18.693594 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.693569    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6e81aea5-openshift-default-dockercfg-9gwq5\""
Oct 31 04:04:18.696448 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696422    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzxd6\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696540 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696469    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696674 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696653    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696738 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696723    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696798 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696766    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696845 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696819    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696923 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696909    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.696987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.696947    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.697047 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.697000    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.798330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.798287    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.798581 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.798559    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.798833    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzxd6\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.805047    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810524    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.805791    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810719    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810798    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810836    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810854    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810864    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.810997    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.811661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.811468    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.812252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.811847    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.814170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.814141    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.814375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.814357    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.869033 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.868958    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzxd6\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.873640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.873609    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token\") pod \"e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") " pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:18.896378 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:18.896349    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:19.005544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.005515    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:19.180986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.180956    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 04:04:19.319463 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.319429    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj\" (UniqueName: \"kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj\") pod \"1acc431a-e60d-48e7-bc7e-035320647d1f\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") "
Oct 31 04:04:19.319652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.319516    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume\") pod \"1acc431a-e60d-48e7-bc7e-035320647d1f\" (UID: \"1acc431a-e60d-48e7-bc7e-035320647d1f\") "
Oct 31 04:04:19.320148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.320117    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume" pod "1acc431a-e60d-48e7-bc7e-035320647d1f" (UID: "1acc431a-e60d-48e7-bc7e-035320647d1f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:04:19.320365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.320344    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1acc431a-e60d-48e7-bc7e-035320647d1f-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:19.322402 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.322372    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj" pod "1acc431a-e60d-48e7-bc7e-035320647d1f" (UID: "1acc431a-e60d-48e7-bc7e-035320647d1f"). InnerVolumeSpecName "kube-api-access-bc4sj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:19.422162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.422073    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc4sj\" (UniqueName: \"kubernetes.io/projected/1acc431a-e60d-48e7-bc7e-035320647d1f-kube-api-access-bc4sj\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:19.435756 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:04:19.435722    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab718b0_7c82_4fdf_bae9_4849a5a537e9.slice/crio-493ecf1013a0955957aa93da9473e149354b66d3876e3037f7080733585b6450 WatchSource:0}: Error finding container 493ecf1013a0955957aa93da9473e149354b66d3876e3037f7080733585b6450: Status 404 returned error can't find the container with id 493ecf1013a0955957aa93da9473e149354b66d3876e3037f7080733585b6450
Oct 31 04:04:19.438711 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.438677    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:04:19.438788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.438748    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:04:19.438823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.438786    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:04:19.439593 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.439539    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"]
Oct 31 04:04:19.796267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.795354    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" event={"ID":"eab718b0-7c82-4fdf-bae9-4849a5a537e9","Type":"ContainerStarted","Data":"beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"}
Oct 31 04:04:19.796267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.795427    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" event={"ID":"eab718b0-7c82-4fdf-bae9-4849a5a537e9","Type":"ContainerStarted","Data":"493ecf1013a0955957aa93da9473e149354b66d3876e3037f7080733585b6450"}
Oct 31 04:04:19.796836 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.796798    5906 generic.go:356] "Generic (PLEG): container finished" podID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerID="691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e" exitCode=0
Oct 31 04:04:19.796895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.796836    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"
Oct 31 04:04:19.796895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.796879    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" event={"ID":"1acc431a-e60d-48e7-bc7e-035320647d1f","Type":"ContainerDied","Data":"691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e"}
Oct 31 04:04:19.797020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.796929    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj" event={"ID":"1acc431a-e60d-48e7-bc7e-035320647d1f","Type":"ContainerDied","Data":"960df353efc0ea8e10d9014b2d910767619940a7ea36d66d1292a8ea7f006dfc"}
Oct 31 04:04:19.797020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.796954    5906 scope.go:122] "RemoveContainer" containerID="691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e"
Oct 31 04:04:19.808216 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.808187    5906 scope.go:122] "RemoveContainer" containerID="691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e"
Oct 31 04:04:19.808536 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:04:19.808503    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e\": container with ID starting with 691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e not found: ID does not exist" containerID="691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e"
Oct 31 04:04:19.808583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.808544    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e"} err="failed to get container status \"691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e\": rpc error: code = NotFound desc = could not find container \"691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e\": container with ID starting with 691ad4d1a625391bfa205c3ae3d649a310c1156b25f572d93fbff35946ee637e not found: ID does not exist"
Oct 31 04:04:19.896456 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.896427    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:19.922954 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:19.922896    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podStartSLOduration=1.9228812720000001 podStartE2EDuration="1.922881272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:04:19.92048233 +0000 UTC m=+2109.754342658" watchObservedRunningTime="2026-10-31 04:04:19.922881272 +0000 UTC m=+2109.756741577"
Oct 31 04:04:20.045345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:20.040473    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"]
Oct 31 04:04:20.071102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:20.066496    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-68aa3e-8594547f55-mv4lj"]
Oct 31 04:04:20.353806 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:20.353725    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" path="/var/lib/kubelet/pods/1acc431a-e60d-48e7-bc7e-035320647d1f/volumes"
Oct 31 04:04:20.902902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:20.902871    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:21.793443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:21.793396    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:21.811434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:21.811406    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:21.812443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:21.812419    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:04:22.614579 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:22.614541    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:22.754935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:22.754900    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:23.201297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:23.201266    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:23.342050 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:23.341996    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:23.505672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:23.505639    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:24.111836 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:24.111801    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:24.890670 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:24.890638    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:25.086369 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.086337    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:25.510953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.508141    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"]
Oct 31 04:04:25.528236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.526556    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerName="payload-pre-processing"
Oct 31 04:04:25.528236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.526596    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerName="payload-pre-processing"
Oct 31 04:04:25.528236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.526902    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1acc431a-e60d-48e7-bc7e-035320647d1f" containerName="payload-pre-processing"
Oct 31 04:04:25.545235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.545185    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.610196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.610115    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-3473a4-openshift-default-dockercfg-s92kd\""
Oct 31 04:04:25.627917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.627883    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.722676    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"]
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726107    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726172    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726260    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726384    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726421    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726487    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726557    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726632    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6qbq\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.726706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.726726    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.843623 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843523    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.843623 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843602    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.843897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843646    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.843897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843703    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.843897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843835    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.843897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843873    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.844120 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.843943    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.844120 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.844019    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.844120 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.844106    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6qbq\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.844683 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.844654    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.845730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.845703    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.846387 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.846358    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.846678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.846629    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.846899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.846878    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.853524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.848089    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.854644 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.854613    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.924005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.919388    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.924005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.921163    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6qbq\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq\") pod \"e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") " pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:25.933324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:25.928452    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:26.040256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.040187    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:26.066774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.066747    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:26.465370 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:04:26.465331    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e688e64_20e8_471d_97b5_55ebadcaa984.slice/crio-b84ce3bea6b41c834a9a3652ade6c7fe93e93f1dbd4267c020bba1d7742dd75e WatchSource:0}: Error finding container b84ce3bea6b41c834a9a3652ade6c7fe93e93f1dbd4267c020bba1d7742dd75e: Status 404 returned error can't find the container with id b84ce3bea6b41c834a9a3652ade6c7fe93e93f1dbd4267c020bba1d7742dd75e
Oct 31 04:04:26.469618 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.469582    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:04:26.469729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.469673    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:04:26.469729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.469699    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:04:26.470263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.470244    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"]
Oct 31 04:04:26.713596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.713489    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:26.903817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.903680    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:26.955309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.946619    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" event={"ID":"5e688e64-20e8-471d-97b5-55ebadcaa984","Type":"ContainerStarted","Data":"ea07630487e90ba96ba551eb33d94857af930af75b8ae58fb4ae59fd213b30b0"}
Oct 31 04:04:26.955309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:26.954874    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" event={"ID":"5e688e64-20e8-471d-97b5-55ebadcaa984","Type":"ContainerStarted","Data":"b84ce3bea6b41c834a9a3652ade6c7fe93e93f1dbd4267c020bba1d7742dd75e"}
Oct 31 04:04:27.041200 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:27.041169    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:27.047214 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:27.047175    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:27.052988 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:27.052964    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:27.074753 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:27.074705    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podStartSLOduration=2.074691591 podStartE2EDuration="2.074691591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:04:27.069819124 +0000 UTC m=+2116.903679430" watchObservedRunningTime="2026-10-31 04:04:27.074691591 +0000 UTC m=+2116.908551895"
Oct 31 04:04:27.962947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:27.962916    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:27.963812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:27.963795    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:04:30.354142 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:30.354102    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:30.499025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:30.498977    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:31.482004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:31.481972    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:31.626506 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:31.626464    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:31.791840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:31.791804    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:36.276877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:36.276845    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:36.425751 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:04:36.425708    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:04:41.125325 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:41.125273    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:43.767085 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:43.767036    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"]
Oct 31 04:04:43.767593 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:43.767278    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" containerID="cri-o://150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30" gracePeriod=30
Oct 31 04:04:45.177479 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.177440    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"]
Oct 31 04:04:45.178303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.177677    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" podUID="852a17b4-91c5-4976-8be8-677229750cf4" containerName="maas-api" containerID="cri-o://7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12" gracePeriod=30
Oct 31 04:04:45.261792 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.261754    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:45.630231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.618580    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:04:45.679582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.677694    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs\") pod \"852a17b4-91c5-4976-8be8-677229750cf4\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") "
Oct 31 04:04:45.679582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.677772    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w\" (UniqueName: \"kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w\") pod \"852a17b4-91c5-4976-8be8-677229750cf4\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") "
Oct 31 04:04:45.679582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.677875    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\") pod \"852a17b4-91c5-4976-8be8-677229750cf4\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") "
Oct 31 04:04:45.679582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.677992    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca\") pod \"852a17b4-91c5-4976-8be8-677229750cf4\" (UID: \"852a17b4-91c5-4976-8be8-677229750cf4\") "
Oct 31 04:04:45.682788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.682754    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca" pod "852a17b4-91c5-4976-8be8-677229750cf4" (UID: "852a17b4-91c5-4976-8be8-677229750cf4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:04:45.704264 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.702505    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls" pod "852a17b4-91c5-4976-8be8-677229750cf4" (UID: "852a17b4-91c5-4976-8be8-677229750cf4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:04:45.709316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.709259    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w" pod "852a17b4-91c5-4976-8be8-677229750cf4" (UID: "852a17b4-91c5-4976-8be8-677229750cf4"). InnerVolumeSpecName "kube-api-access-xcs8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:45.717722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.717684    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs" pod "852a17b4-91c5-4976-8be8-677229750cf4" (UID: "852a17b4-91c5-4976-8be8-677229750cf4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:04:45.784417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.784324    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:45.784417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.784371    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcs8w\" (UniqueName: \"kubernetes.io/projected/852a17b4-91c5-4976-8be8-677229750cf4-kube-api-access-xcs8w\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:45.784417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.784387    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/852a17b4-91c5-4976-8be8-677229750cf4-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:45.784417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.784402    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/852a17b4-91c5-4976-8be8-677229750cf4-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:45.917318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.917282    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:04:45.917470 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:45.917340    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:04:46.195515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.195285    5906 generic.go:356] "Generic (PLEG): container finished" podID="852a17b4-91c5-4976-8be8-677229750cf4" containerID="7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12" exitCode=0
Oct 31 04:04:46.196233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.196190    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" event={"ID":"852a17b4-91c5-4976-8be8-677229750cf4","Type":"ContainerDied","Data":"7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12"}
Oct 31 04:04:46.196447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.196431    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms" event={"ID":"852a17b4-91c5-4976-8be8-677229750cf4","Type":"ContainerDied","Data":"ea08ee773c594f0ecc35a11f8ddb150791783a7006edbe1584f5e53908d24a01"}
Oct 31 04:04:46.196771 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.196646    5906 scope.go:122] "RemoveContainer" containerID="7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12"
Oct 31 04:04:46.196976 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.196752    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"
Oct 31 04:04:46.238389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.238346    5906 scope.go:122] "RemoveContainer" containerID="7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12"
Oct 31 04:04:46.239060 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:04:46.239025    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12\": container with ID starting with 7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12 not found: ID does not exist" containerID="7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12"
Oct 31 04:04:46.239325 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.239271    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12"} err="failed to get container status \"7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12\": rpc error: code = NotFound desc = could not find container \"7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12\": container with ID starting with 7e4d23d15b579921e5c48e8ea97e7c14ee029205dbc0ee3e61a46d43d5397c12 not found: ID does not exist"
Oct 31 04:04:46.249507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.249480    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"]
Oct 31 04:04:46.249744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.249719    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" podUID="295a93f5-e299-4961-876d-d41f4528f5df" containerName="maas-api" containerID="cri-o://b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72" gracePeriod=30
Oct 31 04:04:46.349838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.349804    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:46.537888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.513646    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"]
Oct 31 04:04:46.537888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.517686    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-b4a3a5-57975b4b45-qdmms"]
Oct 31 04:04:46.702963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.681867    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:46.788441 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.786519    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:04:46.827498 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.827463    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs\") pod \"295a93f5-e299-4961-876d-d41f4528f5df\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") "
Oct 31 04:04:46.827708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.827553    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\") pod \"295a93f5-e299-4961-876d-d41f4528f5df\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") "
Oct 31 04:04:46.827781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.827734    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca\") pod \"295a93f5-e299-4961-876d-d41f4528f5df\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") "
Oct 31 04:04:46.827781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.827766    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf\" (UniqueName: \"kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf\") pod \"295a93f5-e299-4961-876d-d41f4528f5df\" (UID: \"295a93f5-e299-4961-876d-d41f4528f5df\") "
Oct 31 04:04:46.828127 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.828095    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca" pod "295a93f5-e299-4961-876d-d41f4528f5df" (UID: "295a93f5-e299-4961-876d-d41f4528f5df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:04:46.828602 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.828575    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295a93f5-e299-4961-876d-d41f4528f5df-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:46.829913 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.829890    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls" pod "295a93f5-e299-4961-876d-d41f4528f5df" (UID: "295a93f5-e299-4961-876d-d41f4528f5df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:04:46.829986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.829958    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf" pod "295a93f5-e299-4961-876d-d41f4528f5df" (UID: "295a93f5-e299-4961-876d-d41f4528f5df"). InnerVolumeSpecName "kube-api-access-djmkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:46.830135 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.830109    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs" pod "295a93f5-e299-4961-876d-d41f4528f5df" (UID: "295a93f5-e299-4961-876d-d41f4528f5df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:04:46.930526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.930472    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:46.930526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.930510    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295a93f5-e299-4961-876d-d41f4528f5df-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:46.930526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.930525    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djmkf\" (UniqueName: \"kubernetes.io/projected/295a93f5-e299-4961-876d-d41f4528f5df-kube-api-access-djmkf\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:46.959313 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:46.959197    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:47.289234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.284811    5906 generic.go:356] "Generic (PLEG): container finished" podID="295a93f5-e299-4961-876d-d41f4528f5df" containerID="b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72" exitCode=0
Oct 31 04:04:47.289234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.284960    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" event={"ID":"295a93f5-e299-4961-876d-d41f4528f5df","Type":"ContainerDied","Data":"b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72"}
Oct 31 04:04:47.289234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.285012    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml" event={"ID":"295a93f5-e299-4961-876d-d41f4528f5df","Type":"ContainerDied","Data":"5c3d068fd5556a36992e5f47834f8921e581858c4cd81fa1945e4b033d9d6048"}
Oct 31 04:04:47.289234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.285040    5906 scope.go:122] "RemoveContainer" containerID="b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72"
Oct 31 04:04:47.289234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.285161    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"
Oct 31 04:04:47.348168 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.348143    5906 scope.go:122] "RemoveContainer" containerID="b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72"
Oct 31 04:04:47.348689 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:04:47.348661    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72\": container with ID starting with b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72 not found: ID does not exist" containerID="b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72"
Oct 31 04:04:47.348795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.348697    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72"} err="failed to get container status \"b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72\": rpc error: code = NotFound desc = could not find container \"b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72\": container with ID starting with b3f98d2c9561301a184d40b05c661ca866bc87b50a039e4aa103a64e7f102e72 not found: ID does not exist"
Oct 31 04:04:47.481027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.480945    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"]
Oct 31 04:04:47.533220 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.533178    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-976cea-78b68b99c4-ggfml"]
Oct 31 04:04:47.612488 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.612445    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:47.880955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:47.880911    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:48.085044 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:48.085011    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:48.234852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:48.224319    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:48.354290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:48.354257    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="295a93f5-e299-4961-876d-d41f4528f5df" path="/var/lib/kubelet/pods/295a93f5-e299-4961-876d-d41f4528f5df/volumes"
Oct 31 04:04:48.354790 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:48.354776    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="852a17b4-91c5-4976-8be8-677229750cf4" path="/var/lib/kubelet/pods/852a17b4-91c5-4976-8be8-677229750cf4/volumes"
Oct 31 04:04:48.866666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:48.866596    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:49.064910 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.064882    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 04:04:49.171162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171121    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.171347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171237    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.171347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171289    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.171442 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171362    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.171504 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171456    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.171633 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171451    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.171757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171614    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.171847 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171660    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.171914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171701    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.171914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171868    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.172021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.171964    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.172086 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172021    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert\") pod \"86373121-d21a-454f-a4b8-d63a27f273f4\" (UID: \"86373121-d21a-454f-a4b8-d63a27f273f4\") "
Oct 31 04:04:49.172485 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172459    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:04:49.172702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172684    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.172917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172899    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.173014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172921    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86373121-d21a-454f-a4b8-d63a27f273f4-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.173014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172936    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.173014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172949    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.173014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.172964    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.173773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.173746    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:49.173972 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.173953    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "kube-api-access-8nvd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:49.174037 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.173973    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.174199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.174180    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo" pod "86373121-d21a-454f-a4b8-d63a27f273f4" (UID: "86373121-d21a-454f-a4b8-d63a27f273f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:04:49.274371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.274334    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nvd5\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-kube-api-access-8nvd5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.274371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.274368    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86373121-d21a-454f-a4b8-d63a27f273f4-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.274597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.274382    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86373121-d21a-454f-a4b8-d63a27f273f4-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.274597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.274395    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86373121-d21a-454f-a4b8-d63a27f273f4-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.334358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.334265    5906 generic.go:356] "Generic (PLEG): container finished" podID="86373121-d21a-454f-a4b8-d63a27f273f4" containerID="150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30" exitCode=0
Oct 31 04:04:49.334358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.334339    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"
Oct 31 04:04:49.334582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.334359    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" event={"ID":"86373121-d21a-454f-a4b8-d63a27f273f4","Type":"ContainerDied","Data":"150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30"}
Oct 31 04:04:49.334582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.334415    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q" event={"ID":"86373121-d21a-454f-a4b8-d63a27f273f4","Type":"ContainerDied","Data":"5c018857fadcd5e182fcca0522bf29c43aec7ca0315d908301772cad430a5dfc"}
Oct 31 04:04:49.334582 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.334444    5906 scope.go:122] "RemoveContainer" containerID="150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30"
Oct 31 04:04:49.361768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.361707    5906 scope.go:122] "RemoveContainer" containerID="150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30"
Oct 31 04:04:49.362395 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:04:49.362365    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30\": container with ID starting with 150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30 not found: ID does not exist" containerID="150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30"
Oct 31 04:04:49.362515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.362398    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30"} err="failed to get container status \"150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30\": rpc error: code = NotFound desc = could not find container \"150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30\": container with ID starting with 150517a0fe362d982630b3ee0dd76ca87a5c5dae40f5c2bc655b2bda4ff53f30 not found: ID does not exist"
Oct 31 04:04:49.506300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.506256    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"]
Oct 31 04:04:49.510380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:49.510338    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-7gs5q"]
Oct 31 04:04:50.010683 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:50.010642    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:50.265812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:50.265684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:50.353719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:50.353687    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" path="/var/lib/kubelet/pods/86373121-d21a-454f-a4b8-d63a27f273f4/volumes"
Oct 31 04:04:50.408617 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:50.408580    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:53.004111 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:53.004081    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:53.225278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:53.224359    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:53.399571 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:53.399493    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:54.136437 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:54.136404    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:54.291229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:54.290841    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:54.470245 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:54.470123    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:55.554620 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:55.554581    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:55.700133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:55.700094    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:04:58.053762 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:04:58.053730    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:02.840065 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:02.840032    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:05.887813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:05.887777    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"]
Oct 31 04:05:05.888658 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:05.888604    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" containerID="cri-o://3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef" gracePeriod=30
Oct 31 04:05:05.889955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:05.889862    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:06.384590 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:06.384547    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:05:07.603465 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:07.603428    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:07.945173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:07.945074    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"]
Oct 31 04:05:08.047062 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:08.047020    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"]
Oct 31 04:05:08.277677 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:08.277643    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:08.418234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:08.418161    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:11.059786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.059753    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused" start-of-body=
Oct 31 04:05:11.060235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.059809    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.105:15021/healthz/ready\": dial tcp 10.134.0.105:15021: connect: connection refused"
Oct 31 04:05:11.174147 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.174079    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 04:05:11.189326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.189295    5906 scope.go:122] "RemoveContainer" containerID="0087d8aa6f291bfe325aae3f6ffff0c8cb85c62bb30e1cde9c7d8e711919175a"
Oct 31 04:05:11.253959 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.253926    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254000    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254034    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254059    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254121    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254146    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254271    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254315    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254360    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4\") pod \"749d4774-58db-40e5-bf02-cf4d37da86dc\" (UID: \"749d4774-58db-40e5-bf02-cf4d37da86dc\") "
Oct 31 04:05:11.254645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254433    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:11.254645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254480    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:11.254759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254650    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:11.254759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254687    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:11.254759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.254706    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:11.255052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.255024    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.255142 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.255055    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.255142 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.255066    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.255142 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.255075    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/749d4774-58db-40e5-bf02-cf4d37da86dc-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.255142 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.255083    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.256385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.256359    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:05:11.256704 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.256681    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:11.256802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.256736    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:11.256848 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.256815    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4" pod "749d4774-58db-40e5-bf02-cf4d37da86dc" (UID: "749d4774-58db-40e5-bf02-cf4d37da86dc"). InnerVolumeSpecName "kube-api-access-jhpd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:11.356478 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.356445    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.356478 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.356476    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.356478 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.356486    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/749d4774-58db-40e5-bf02-cf4d37da86dc-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.356712 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.356497    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhpd4\" (UniqueName: \"kubernetes.io/projected/749d4774-58db-40e5-bf02-cf4d37da86dc-kube-api-access-jhpd4\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:11.589253 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.589196    5906 generic.go:356] "Generic (PLEG): container finished" podID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerID="3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef" exitCode=0
Oct 31 04:05:11.589469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.589279    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"
Oct 31 04:05:11.589469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.589283    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" event={"ID":"749d4774-58db-40e5-bf02-cf4d37da86dc","Type":"ContainerDied","Data":"3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef"}
Oct 31 04:05:11.589469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.589344    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn" event={"ID":"749d4774-58db-40e5-bf02-cf4d37da86dc","Type":"ContainerDied","Data":"88e9c63033f72dfe2fa708d01df85abda3de84c479dbdf3c2f047d67ac691f04"}
Oct 31 04:05:11.589469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.589370    5906 scope.go:122] "RemoveContainer" containerID="3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef"
Oct 31 04:05:11.600404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.600382    5906 scope.go:122] "RemoveContainer" containerID="3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef"
Oct 31 04:05:11.600682 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:11.600660    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef\": container with ID starting with 3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef not found: ID does not exist" containerID="3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef"
Oct 31 04:05:11.600764 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.600690    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef"} err="failed to get container status \"3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef\": rpc error: code = NotFound desc = could not find container \"3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef\": container with ID starting with 3073804664a4973b6f084f2be17d95ae81f3cdce52d1e59a26ca466c0dfca8ef not found: ID does not exist"
Oct 31 04:05:11.830109 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.830069    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"]
Oct 31 04:05:11.833524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:11.833494    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-976cea-openshift-default-85b4bff6f9-v6jzn"]
Oct 31 04:05:12.354542 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:12.354510    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" path="/var/lib/kubelet/pods/749d4774-58db-40e5-bf02-cf4d37da86dc/volumes"
Oct 31 04:05:12.379314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:12.379282    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:12.946031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:12.945987    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7" podUID="bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" containerName="payload-processing" containerID="cri-o://3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef" gracePeriod=25
Oct 31 04:05:13.047336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.047300    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn" podUID="32753339-e8f8-4629-88da-e6ba8ffb6180" containerName="payload-pre-processing" containerID="cri-o://b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71" gracePeriod=25
Oct 31 04:05:13.261121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.261087    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:05:13.342362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.342339    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:05:13.386640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.386610    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb\" (UniqueName: \"kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb\") pod \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") "
Oct 31 04:05:13.387035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.386701    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume\") pod \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\" (UID: \"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1\") "
Oct 31 04:05:13.387144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.387117    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume" pod "bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" (UID: "bf4c0fc0-96a8-4bda-847c-9652c1c44aa1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:13.387465 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.387442    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:13.388842 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.388810    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb" pod "bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" (UID: "bf4c0fc0-96a8-4bda-847c-9652c1c44aa1"). InnerVolumeSpecName "kube-api-access-627tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:13.488009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.487924    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume\") pod \"32753339-e8f8-4629-88da-e6ba8ffb6180\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") "
Oct 31 04:05:13.488170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.488145    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn\" (UniqueName: \"kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn\") pod \"32753339-e8f8-4629-88da-e6ba8ffb6180\" (UID: \"32753339-e8f8-4629-88da-e6ba8ffb6180\") "
Oct 31 04:05:13.488423 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.488399    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume" pod "32753339-e8f8-4629-88da-e6ba8ffb6180" (UID: "32753339-e8f8-4629-88da-e6ba8ffb6180"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:13.488797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.488774    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-627tb\" (UniqueName: \"kubernetes.io/projected/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1-kube-api-access-627tb\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:13.488797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.488797    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32753339-e8f8-4629-88da-e6ba8ffb6180-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:13.490320 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.490297    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn" pod "32753339-e8f8-4629-88da-e6ba8ffb6180" (UID: "32753339-e8f8-4629-88da-e6ba8ffb6180"). InnerVolumeSpecName "kube-api-access-mrftn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:13.589930 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.589893    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrftn\" (UniqueName: \"kubernetes.io/projected/32753339-e8f8-4629-88da-e6ba8ffb6180-kube-api-access-mrftn\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:13.607758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.607727    5906 generic.go:356] "Generic (PLEG): container finished" podID="bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" containerID="3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef" exitCode=0
Oct 31 04:05:13.607926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.607774    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"
Oct 31 04:05:13.607926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.607808    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7" event={"ID":"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1","Type":"ContainerDied","Data":"3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef"}
Oct 31 04:05:13.607926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.607879    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7" event={"ID":"bf4c0fc0-96a8-4bda-847c-9652c1c44aa1","Type":"ContainerDied","Data":"1adee5db877b37d80c6b8249f04b6d7868388ad194e2780ddb3c28a0dd86c69e"}
Oct 31 04:05:13.607926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.607914    5906 scope.go:122] "RemoveContainer" containerID="3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef"
Oct 31 04:05:13.609289 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.609264    5906 generic.go:356] "Generic (PLEG): container finished" podID="32753339-e8f8-4629-88da-e6ba8ffb6180" containerID="b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71" exitCode=0
Oct 31 04:05:13.609370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.609329    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"
Oct 31 04:05:13.609370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.609354    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn" event={"ID":"32753339-e8f8-4629-88da-e6ba8ffb6180","Type":"ContainerDied","Data":"b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71"}
Oct 31 04:05:13.609468 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.609404    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn" event={"ID":"32753339-e8f8-4629-88da-e6ba8ffb6180","Type":"ContainerDied","Data":"f40e0eb7800109a29364c8e97b3a277906df0c25a0b50323750e39ba15d15f1d"}
Oct 31 04:05:13.620113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.620083    5906 scope.go:122] "RemoveContainer" containerID="3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef"
Oct 31 04:05:13.620442 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:13.620424    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef\": container with ID starting with 3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef not found: ID does not exist" containerID="3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef"
Oct 31 04:05:13.620495 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.620448    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef"} err="failed to get container status \"3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef\": rpc error: code = NotFound desc = could not find container \"3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef\": container with ID starting with 3437f619ca91691269b9a362ef3443252ad1d1232d07c8b24c1a28a0987e83ef not found: ID does not exist"
Oct 31 04:05:13.620495 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.620466    5906 scope.go:122] "RemoveContainer" containerID="b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71"
Oct 31 04:05:13.630028 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.630008    5906 scope.go:122] "RemoveContainer" containerID="b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71"
Oct 31 04:05:13.630355 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:13.630332    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71\": container with ID starting with b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71 not found: ID does not exist" containerID="b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71"
Oct 31 04:05:13.630448 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.630361    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71"} err="failed to get container status \"b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71\": rpc error: code = NotFound desc = could not find container \"b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71\": container with ID starting with b13ec0b9176b16d16c4400181edc8e2a4403f9a00ab09ae33e88d899fb543b71 not found: ID does not exist"
Oct 31 04:05:13.782326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.782288    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"]
Oct 31 04:05:13.786331 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.786301    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-f9a7fa-844b89bcc5-2vwgn"]
Oct 31 04:05:13.899893 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.899853    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"]
Oct 31 04:05:13.903838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:13.903809    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-f9a7fa-7498fc4cbf-k4np7"]
Oct 31 04:05:14.354444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:14.354409    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32753339-e8f8-4629-88da-e6ba8ffb6180" path="/var/lib/kubelet/pods/32753339-e8f8-4629-88da-e6ba8ffb6180/volumes"
Oct 31 04:05:14.354902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:14.354888    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" path="/var/lib/kubelet/pods/bf4c0fc0-96a8-4bda-847c-9652c1c44aa1/volumes"
Oct 31 04:05:17.367237 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:17.366953    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:18.179711 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:18.179676    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"]
Oct 31 04:05:18.296060 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:18.291748    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"]
Oct 31 04:05:18.518099 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:18.518069    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:18.676275 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:18.676114    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:22.190860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:22.190814    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:23.180334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.180259    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m" podUID="c68df62e-2efc-4571-8f12-f68b978d8407" containerName="payload-processing" containerID="cri-o://590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30" gracePeriod=25
Oct 31 04:05:23.293021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.292973    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl" podUID="83dec427-47e3-49f6-8cf3-134b41131e40" containerName="payload-pre-processing" containerID="cri-o://f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725" gracePeriod=25
Oct 31 04:05:23.496317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.496288    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:05:23.524449 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.524416    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume\") pod \"c68df62e-2efc-4571-8f12-f68b978d8407\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") "
Oct 31 04:05:23.524624 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.524492    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq\" (UniqueName: \"kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq\") pod \"c68df62e-2efc-4571-8f12-f68b978d8407\" (UID: \"c68df62e-2efc-4571-8f12-f68b978d8407\") "
Oct 31 04:05:23.524884 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.524854    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume" pod "c68df62e-2efc-4571-8f12-f68b978d8407" (UID: "c68df62e-2efc-4571-8f12-f68b978d8407"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:23.525451 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.525429    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68df62e-2efc-4571-8f12-f68b978d8407-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:23.528014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.527963    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq" pod "c68df62e-2efc-4571-8f12-f68b978d8407" (UID: "c68df62e-2efc-4571-8f12-f68b978d8407"). InnerVolumeSpecName "kube-api-access-9flgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:23.587264 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.587234    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:05:23.626349 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.626317    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume\") pod \"83dec427-47e3-49f6-8cf3-134b41131e40\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") "
Oct 31 04:05:23.626535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.626389    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8\" (UniqueName: \"kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8\") pod \"83dec427-47e3-49f6-8cf3-134b41131e40\" (UID: \"83dec427-47e3-49f6-8cf3-134b41131e40\") "
Oct 31 04:05:23.626809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.626778    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume" pod "83dec427-47e3-49f6-8cf3-134b41131e40" (UID: "83dec427-47e3-49f6-8cf3-134b41131e40"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:23.627268 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.627194    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9flgq\" (UniqueName: \"kubernetes.io/projected/c68df62e-2efc-4571-8f12-f68b978d8407-kube-api-access-9flgq\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:23.627374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.627278    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83dec427-47e3-49f6-8cf3-134b41131e40-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:23.628745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.628714    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8" pod "83dec427-47e3-49f6-8cf3-134b41131e40" (UID: "83dec427-47e3-49f6-8cf3-134b41131e40"). InnerVolumeSpecName "kube-api-access-xftd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:23.728397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.728303    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xftd8\" (UniqueName: \"kubernetes.io/projected/83dec427-47e3-49f6-8cf3-134b41131e40-kube-api-access-xftd8\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:23.770816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.770762    5906 generic.go:356] "Generic (PLEG): container finished" podID="83dec427-47e3-49f6-8cf3-134b41131e40" containerID="f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725" exitCode=0
Oct 31 04:05:23.770816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.770805    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl" event={"ID":"83dec427-47e3-49f6-8cf3-134b41131e40","Type":"ContainerDied","Data":"f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725"}
Oct 31 04:05:23.771277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.770876    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl" event={"ID":"83dec427-47e3-49f6-8cf3-134b41131e40","Type":"ContainerDied","Data":"77e3363eb3f00bea22cc85f300749d3ec1c3c931a8a4f959a504fd4353b6bc37"}
Oct 31 04:05:23.771277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.770818    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"
Oct 31 04:05:23.771277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.770910    5906 scope.go:122] "RemoveContainer" containerID="f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725"
Oct 31 04:05:23.774079 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.774051    5906 generic.go:356] "Generic (PLEG): container finished" podID="c68df62e-2efc-4571-8f12-f68b978d8407" containerID="590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30" exitCode=0
Oct 31 04:05:23.774271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.774162    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m" event={"ID":"c68df62e-2efc-4571-8f12-f68b978d8407","Type":"ContainerDied","Data":"590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30"}
Oct 31 04:05:23.774271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.774226    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m" event={"ID":"c68df62e-2efc-4571-8f12-f68b978d8407","Type":"ContainerDied","Data":"78dd27e5a73ff1617402f3ab24791d26651549d1473c9b64d7ee6fc7c1335cf0"}
Oct 31 04:05:23.774271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.774232    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"
Oct 31 04:05:23.786268 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.786104    5906 scope.go:122] "RemoveContainer" containerID="f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725"
Oct 31 04:05:23.786676 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:23.786646    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725\": container with ID starting with f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725 not found: ID does not exist" containerID="f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725"
Oct 31 04:05:23.786802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.786684    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725"} err="failed to get container status \"f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725\": rpc error: code = NotFound desc = could not find container \"f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725\": container with ID starting with f663f8a96bc805d6e62659c503da7d135623195f4cc8a656b96c32515c81b725 not found: ID does not exist"
Oct 31 04:05:23.786802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.786708    5906 scope.go:122] "RemoveContainer" containerID="590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30"
Oct 31 04:05:23.801664 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.801640    5906 scope.go:122] "RemoveContainer" containerID="590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30"
Oct 31 04:05:23.801983 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:23.801962    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30\": container with ID starting with 590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30 not found: ID does not exist" containerID="590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30"
Oct 31 04:05:23.802052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.801995    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30"} err="failed to get container status \"590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30\": rpc error: code = NotFound desc = could not find container \"590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30\": container with ID starting with 590e4d2eebc0d4e79130d1e3f7a7dd6e8d2010e262cee665ed962fd0da5c8c30 not found: ID does not exist"
Oct 31 04:05:23.944630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.944591    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"]
Oct 31 04:05:23.948154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:23.948119    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-833977-5f65fbdcf-6xw8m"]
Oct 31 04:05:24.061875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:24.061840    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"]
Oct 31 04:05:24.066036 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:24.066004    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-833977-6997f9dd8c-ngvhl"]
Oct 31 04:05:24.356837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:24.356743    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83dec427-47e3-49f6-8cf3-134b41131e40" path="/var/lib/kubelet/pods/83dec427-47e3-49f6-8cf3-134b41131e40/volumes"
Oct 31 04:05:24.357450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:24.357429    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c68df62e-2efc-4571-8f12-f68b978d8407" path="/var/lib/kubelet/pods/c68df62e-2efc-4571-8f12-f68b978d8407/volumes"
Oct 31 04:05:25.793550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:25.793515    5906 generic.go:356] "Generic (PLEG): container finished" podID="856a336a-5439-48b5-af95-01572728f411" containerID="afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845" exitCode=0
Oct 31 04:05:25.794046 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:25.793589    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" event={"ID":"856a336a-5439-48b5-af95-01572728f411","Type":"ContainerDied","Data":"afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"}
Oct 31 04:05:25.794097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:25.794080    5906 scope.go:122] "RemoveContainer" containerID="afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"
Oct 31 04:05:25.795912 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:25.795880    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:05:25.796005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:25.795957    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:05:25.796005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:25.795991    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:05:26.375706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:26.375662    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:26.688251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:26.688122    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:26.823255 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:26.823200    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" event={"ID":"856a336a-5439-48b5-af95-01572728f411","Type":"ContainerStarted","Data":"693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec"}
Oct 31 04:05:27.136895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:27.136850    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:05:27.137116 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:27.137092    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 04:05:27.137172 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:27.137145    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 04:05:28.137061 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:28.137024    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 04:05:28.137545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:28.137092    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 04:05:29.095762 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:29.095707    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:29.136821 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:29.136782    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 04:05:29.136999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:29.136838    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 04:05:30.137590 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:30.137547    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 04:05:30.138005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:30.137614    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 04:05:31.068412 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.068373    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"]
Oct 31 04:05:31.069184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069161    5906 cpu_manager.go:414] "containerMap: removing container" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy"
Oct 31 04:05:31.069184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069184    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069197    5906 cpu_manager.go:414] "containerMap: removing container" podUID="852a17b4-91c5-4976-8be8-677229750cf4" containerName="maas-api"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069221    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="852a17b4-91c5-4976-8be8-677229750cf4" containerName="maas-api"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069247    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c68df62e-2efc-4571-8f12-f68b978d8407" containerName="payload-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069256    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c68df62e-2efc-4571-8f12-f68b978d8407" containerName="payload-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069303    5906 cpu_manager.go:414] "containerMap: removing container" podUID="32753339-e8f8-4629-88da-e6ba8ffb6180" containerName="payload-pre-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069310    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32753339-e8f8-4629-88da-e6ba8ffb6180" containerName="payload-pre-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069323    5906 cpu_manager.go:414] "containerMap: removing container" podUID="bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" containerName="payload-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069330    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" containerName="payload-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069346    5906 cpu_manager.go:414] "containerMap: removing container" podUID="295a93f5-e299-4961-876d-d41f4528f5df" containerName="maas-api"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069353    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="295a93f5-e299-4961-876d-d41f4528f5df" containerName="maas-api"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069367    5906 cpu_manager.go:414] "containerMap: removing container" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069374    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069386    5906 cpu_manager.go:414] "containerMap: removing container" podUID="83dec427-47e3-49f6-8cf3-134b41131e40" containerName="payload-pre-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069394    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83dec427-47e3-49f6-8cf3-134b41131e40" containerName="payload-pre-processing"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069407    5906 cpu_manager.go:414] "containerMap: removing container" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy"
Oct 31 04:05:31.069426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069414    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069598    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="86373121-d21a-454f-a4b8-d63a27f273f4" containerName="istio-proxy"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069611    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="852a17b4-91c5-4976-8be8-677229750cf4" containerName="maas-api"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069626    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c68df62e-2efc-4571-8f12-f68b978d8407" containerName="payload-processing"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069647    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="32753339-e8f8-4629-88da-e6ba8ffb6180" containerName="payload-pre-processing"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069658    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf4c0fc0-96a8-4bda-847c-9652c1c44aa1" containerName="payload-processing"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069667    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="295a93f5-e299-4961-876d-d41f4528f5df" containerName="maas-api"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069678    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069688    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="83dec427-47e3-49f6-8cf3-134b41131e40" containerName="payload-pre-processing"
Oct 31 04:05:31.070194 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.069700    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="749d4774-58db-40e5-bf02-cf4d37da86dc" containerName="istio-proxy"
Oct 31 04:05:31.073973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.073950    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.118832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.118798    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"]
Oct 31 04:05:31.131597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.131565    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6e81aea5-serving-cert\""
Oct 31 04:05:31.135350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.135298    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:31.137339 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.137312    5906 patch_prober.go:29] interesting pod/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Oct 31 04:05:31.137426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.137381    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Oct 31 04:05:31.169259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.153351    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.169259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.153545    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.169259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.153610    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4279x\" (UniqueName: \"kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.169259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.153637    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.266233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.265516    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.266233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.265599    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4279x\" (UniqueName: \"kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.266233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.265628    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.266233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.265836    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.267089 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.267055    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.270493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.270464    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.270611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.270464    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.287062 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.287032    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:31.330330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.330248    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4279x\" (UniqueName: \"kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.409247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.400510    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:31.467233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.464144    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:31.855703 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:05:31.855670    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc17dd46_e662_49bc_8994_a73af37493a3.slice/crio-a1e42c731e7269e01183dab7b669cd16fab76a196c66dc0bb0bc234358f12164 WatchSource:0}: Error finding container a1e42c731e7269e01183dab7b669cd16fab76a196c66dc0bb0bc234358f12164: Status 404 returned error can't find the container with id a1e42c731e7269e01183dab7b669cd16fab76a196c66dc0bb0bc234358f12164
Oct 31 04:05:31.857445 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.857421    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"]
Oct 31 04:05:31.884377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:31.884339    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl" event={"ID":"bc17dd46-e662-49bc-8994-a73af37493a3","Type":"ContainerStarted","Data":"a1e42c731e7269e01183dab7b669cd16fab76a196c66dc0bb0bc234358f12164"}
Oct 31 04:05:32.141782 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:32.141699    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:05:32.142089 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:32.142055    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:05:32.142948 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:32.142926    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:05:32.895427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:32.895393    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl" event={"ID":"bc17dd46-e662-49bc-8994-a73af37493a3","Type":"ContainerStarted","Data":"4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4"}
Oct 31 04:05:32.895967 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:32.895936    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:33.035989 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:33.035931    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl" podStartSLOduration=1.090126058 podStartE2EDuration="2.035915372s" totalImagesPullingTime="945.789314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:05:33.033912178 +0000 UTC m=+2182.867772486" watchObservedRunningTime="2026-10-31 04:05:33.035915372 +0000 UTC m=+2182.869775678"
Oct 31 04:05:35.627935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:35.627898    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:35.776370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:35.776336    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:36.382597 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:36.382558    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:05:37.525671 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:37.525637    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:37.669091 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:37.669059    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:38.931306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:38.931275    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:05:40.479947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:40.479914    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:43.462243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:43.462176    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"]
Oct 31 04:05:43.462656 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:43.462416    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" containerID="cri-o://f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6" gracePeriod=30
Oct 31 04:05:45.104748 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:45.104713    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:47.026982 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:47.026949    5906 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:05:47.027487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:47.027007    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:05:48.528964 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.528936    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:48.773800 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.773773    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 04:05:48.787521 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787491    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.787679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787540    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.787679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787608    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.787795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787689    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.787795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787724    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.787795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787777    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.787953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787877    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.788023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787896    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:48.788023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787999    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:48.788023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.787979    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:48.788023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788008    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.788366 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788119    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy\") pod \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\" (UID: \"8bfddfc3-1b1a-4166-b8da-181cfd6538cb\") "
Oct 31 04:05:48.788366 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788242    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:48.788366 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788254    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:48.788955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788927    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.789055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788961    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.789055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788977    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.789055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.788991    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.789055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.789004    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.789941 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.789909    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:48.789941 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.789927    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "kube-api-access-wwpqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:48.790103 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.789996    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:05:48.790240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.790195    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy" pod "8bfddfc3-1b1a-4166-b8da-181cfd6538cb" (UID: "8bfddfc3-1b1a-4166-b8da-181cfd6538cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:48.890029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.889995    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.890029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.890025    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.890029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.890036    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwpqx\" (UniqueName: \"kubernetes.io/projected/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-kube-api-access-wwpqx\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:48.890336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:48.890045    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bfddfc3-1b1a-4166-b8da-181cfd6538cb-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.037471 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.037385    5906 generic.go:356] "Generic (PLEG): container finished" podID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerID="f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6" exitCode=0
Oct 31 04:05:49.037471 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.037434    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"
Oct 31 04:05:49.037655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.037469    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" event={"ID":"8bfddfc3-1b1a-4166-b8da-181cfd6538cb","Type":"ContainerDied","Data":"f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6"}
Oct 31 04:05:49.037655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.037525    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb" event={"ID":"8bfddfc3-1b1a-4166-b8da-181cfd6538cb","Type":"ContainerDied","Data":"f154dfd430b44480e36ef54ad9b8553c3198e15fb3377973429bbfe5055d1499"}
Oct 31 04:05:49.037655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.037547    5906 scope.go:122] "RemoveContainer" containerID="f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6"
Oct 31 04:05:49.049812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.049788    5906 scope.go:122] "RemoveContainer" containerID="f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6"
Oct 31 04:05:49.050145 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:05:49.050116    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6\": container with ID starting with f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6 not found: ID does not exist" containerID="f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6"
Oct 31 04:05:49.050317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.050155    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6"} err="failed to get container status \"f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6\": rpc error: code = NotFound desc = could not find container \"f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6\": container with ID starting with f7fbc8da276c728addfac3ea0e9fcbd97fa59daf3783f72dc02e7cecdf8f88a6 not found: ID does not exist"
Oct 31 04:05:49.211901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.211872    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"]
Oct 31 04:05:49.216907 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.216881    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-9c9lb"]
Oct 31 04:05:49.944914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:49.944882    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:50.353864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:50.353828    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" path="/var/lib/kubelet/pods/8bfddfc3-1b1a-4166-b8da-181cfd6538cb/volumes"
Oct 31 04:05:54.750543 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:54.750502    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:57.703165 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:57.703123    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:57.854172 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:57.854137    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:59.440926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:59.440887    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:05:59.934529 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:59.934439    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 31 04:05:59.966508 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:05:59.966468    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:06:00.028482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:00.028379    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:06:00.869401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:00.869361    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:01.072638 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:01.072600    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:01.217499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:01.217405    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:01.822970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:01.822932    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:05.880021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:05.879920    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:06.069638 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:06.069594    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:06.229176 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:06.229038    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:06.384431 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:06:06.384380    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:06:07.856524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:07.853367    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:06:07.856524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:07.853450    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:06:07.865383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:07.865337    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:08.058141 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:08.058105    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:08.198865 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:08.198773    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:09.443806 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.443716    5906 generic.go:356] "Generic (PLEG): container finished" podID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerID="beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed" exitCode=0
Oct 31 04:06:09.443806 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.443788    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" event={"ID":"eab718b0-7c82-4fdf-bae9-4849a5a537e9","Type":"ContainerDied","Data":"beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"}
Oct 31 04:06:09.444410 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.444286    5906 scope.go:122] "RemoveContainer" containerID="beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"
Oct 31 04:06:09.445968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.445938    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:06:09.446068 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.446012    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:06:09.446068 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.446046    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:06:09.964233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.959338    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"]
Oct 31 04:06:09.964233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:09.959944    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" podUID="9e2ce747-c974-460c-b44d-c4e6f0c62734" containerName="maas-api" containerID="cri-o://aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755" gracePeriod=30
Oct 31 04:06:10.052431 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.052400    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:10.359239 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.346429    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:06:10.427940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.427903    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5\" (UniqueName: \"kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5\") pod \"9e2ce747-c974-460c-b44d-c4e6f0c62734\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") "
Oct 31 04:06:10.428259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.428234    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca\") pod \"9e2ce747-c974-460c-b44d-c4e6f0c62734\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") "
Oct 31 04:06:10.428622 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.428600    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs\") pod \"9e2ce747-c974-460c-b44d-c4e6f0c62734\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") "
Oct 31 04:06:10.428871 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.428840    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\") pod \"9e2ce747-c974-460c-b44d-c4e6f0c62734\" (UID: \"9e2ce747-c974-460c-b44d-c4e6f0c62734\") "
Oct 31 04:06:10.434142 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.433816    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca" pod "9e2ce747-c974-460c-b44d-c4e6f0c62734" (UID: "9e2ce747-c974-460c-b44d-c4e6f0c62734"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:10.458265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.456702    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls" pod "9e2ce747-c974-460c-b44d-c4e6f0c62734" (UID: "9e2ce747-c974-460c-b44d-c4e6f0c62734"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:10.458661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.458178    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs" pod "9e2ce747-c974-460c-b44d-c4e6f0c62734" (UID: "9e2ce747-c974-460c-b44d-c4e6f0c62734"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:10.458661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.458370    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5" pod "9e2ce747-c974-460c-b44d-c4e6f0c62734" (UID: "9e2ce747-c974-460c-b44d-c4e6f0c62734"). InnerVolumeSpecName "kube-api-access-kkdq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:10.466217 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.466182    5906 generic.go:356] "Generic (PLEG): container finished" podID="9e2ce747-c974-460c-b44d-c4e6f0c62734" containerID="aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755" exitCode=0
Oct 31 04:06:10.466348 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.466296    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"
Oct 31 04:06:10.477772 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.477739    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" event={"ID":"eab718b0-7c82-4fdf-bae9-4849a5a537e9","Type":"ContainerStarted","Data":"71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6"}
Oct 31 04:06:10.477908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.477800    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" event={"ID":"9e2ce747-c974-460c-b44d-c4e6f0c62734","Type":"ContainerDied","Data":"aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755"}
Oct 31 04:06:10.477908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.477831    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964" event={"ID":"9e2ce747-c974-460c-b44d-c4e6f0c62734","Type":"ContainerDied","Data":"f2c5c1b23dd441d741ffe613e121cfc7c7b463b602da6a3b113bc145f10ac933"}
Oct 31 04:06:10.477908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.477860    5906 scope.go:122] "RemoveContainer" containerID="aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755"
Oct 31 04:06:10.491434 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.491400    5906 scope.go:122] "RemoveContainer" containerID="aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755"
Oct 31 04:06:10.493407 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:06:10.492177    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755\": container with ID starting with aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755 not found: ID does not exist" containerID="aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755"
Oct 31 04:06:10.493407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.492330    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755"} err="failed to get container status \"aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755\": rpc error: code = NotFound desc = could not find container \"aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755\": container with ID starting with aca4cd41066ca86f368358637710db65e7e625a2922451a3e98fc475c2f19755 not found: ID does not exist"
Oct 31 04:06:10.531241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.531109    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:10.531241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.531146    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkdq5\" (UniqueName: \"kubernetes.io/projected/9e2ce747-c974-460c-b44d-c4e6f0c62734-kube-api-access-kkdq5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:10.531241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.531162    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e2ce747-c974-460c-b44d-c4e6f0c62734-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:10.531241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.531175    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e2ce747-c974-460c-b44d-c4e6f0c62734-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:10.604186 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.604145    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"]
Oct 31 04:06:10.614131 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.614096    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-f9a7fa-7b94dcb64-tb964"]
Oct 31 04:06:10.904290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.903345    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:06:10.904290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.903516    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 04:06:10.904290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:10.903573    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 04:06:11.477011 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:11.476973    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:11.722924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:11.722838    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:11.882231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:11.877442    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:11.896828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:11.896795    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 04:06:11.897113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:11.897083    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 04:06:12.374621 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:12.374585    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e2ce747-c974-460c-b44d-c4e6f0c62734" path="/var/lib/kubelet/pods/9e2ce747-c974-460c-b44d-c4e6f0c62734/volumes"
Oct 31 04:06:12.390722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:12.390678    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:12.897685 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:12.897646    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 04:06:12.898103 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:12.897710    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 04:06:13.896940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:13.896906    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 04:06:13.897128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:13.896957    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 04:06:14.897075 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:14.897036    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 04:06:14.897544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:14.897096    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 04:06:15.897196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:15.897157    5906 patch_prober.go:29] interesting pod/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 31 04:06:15.897666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:15.897246    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 31 04:06:16.539941 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:16.539902    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:16.901707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:16.901630    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:06:16.902119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:16.901995    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:06:16.902676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:16.902660    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:06:18.991764 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:18.991732    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:19.136596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:19.136560    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:19.333672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:19.333564    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"]
Oct 31 04:06:19.333901 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:19.333855    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy" containerID="cri-o://703014f2811c89030c1944d1070c7d8f67e2294891d815fa996344efd7b4ba64" gracePeriod=30
Oct 31 04:06:19.342627 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:19.342601    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:19.825273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:19.825236    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:20.677045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:20.677006    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"]
Oct 31 04:06:20.677621 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:20.677263    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" podUID="20f715b5-f7d0-440c-9401-dc9cf82944e9" containerName="maas-api" containerID="cri-o://603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b" gracePeriod=30
Oct 31 04:06:20.771153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:20.771122    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:21.011598 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.011570    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:06:21.065417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.065380    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\") pod \"20f715b5-f7d0-440c-9401-dc9cf82944e9\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") "
Oct 31 04:06:21.065818 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.065447    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs\") pod \"20f715b5-f7d0-440c-9401-dc9cf82944e9\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") "
Oct 31 04:06:21.065818 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.065671    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca\") pod \"20f715b5-f7d0-440c-9401-dc9cf82944e9\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") "
Oct 31 04:06:21.065818 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.065779    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8\" (UniqueName: \"kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8\") pod \"20f715b5-f7d0-440c-9401-dc9cf82944e9\" (UID: \"20f715b5-f7d0-440c-9401-dc9cf82944e9\") "
Oct 31 04:06:21.066108 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.066062    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca" pod "20f715b5-f7d0-440c-9401-dc9cf82944e9" (UID: "20f715b5-f7d0-440c-9401-dc9cf82944e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:21.066557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.066534    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20f715b5-f7d0-440c-9401-dc9cf82944e9-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:21.067822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.067794    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls" pod "20f715b5-f7d0-440c-9401-dc9cf82944e9" (UID: "20f715b5-f7d0-440c-9401-dc9cf82944e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:21.067917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.067832    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs" pod "20f715b5-f7d0-440c-9401-dc9cf82944e9" (UID: "20f715b5-f7d0-440c-9401-dc9cf82944e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:21.067988 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.067963    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8" pod "20f715b5-f7d0-440c-9401-dc9cf82944e9" (UID: "20f715b5-f7d0-440c-9401-dc9cf82944e9"). InnerVolumeSpecName "kube-api-access-nq4r8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:21.179229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.168124    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq4r8\" (UniqueName: \"kubernetes.io/projected/20f715b5-f7d0-440c-9401-dc9cf82944e9-kube-api-access-nq4r8\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:21.179229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.168162    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:21.179229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.168180    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20f715b5-f7d0-440c-9401-dc9cf82944e9-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:21.559040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.559006    5906 generic.go:356] "Generic (PLEG): container finished" podID="20f715b5-f7d0-440c-9401-dc9cf82944e9" containerID="603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b" exitCode=0
Oct 31 04:06:21.559292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.559062    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"
Oct 31 04:06:21.559292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.559096    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" event={"ID":"20f715b5-f7d0-440c-9401-dc9cf82944e9","Type":"ContainerDied","Data":"603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b"}
Oct 31 04:06:21.559292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.559156    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q" event={"ID":"20f715b5-f7d0-440c-9401-dc9cf82944e9","Type":"ContainerDied","Data":"14538c63440a3501c311a5288e0ebda134cf4d0895e13b715e1f4df711f766ac"}
Oct 31 04:06:21.559292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.559186    5906 scope.go:122] "RemoveContainer" containerID="603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b"
Oct 31 04:06:21.572020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.571998    5906 scope.go:122] "RemoveContainer" containerID="603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b"
Oct 31 04:06:21.572391 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:06:21.572368    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b\": container with ID starting with 603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b not found: ID does not exist" containerID="603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b"
Oct 31 04:06:21.572457 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.572403    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b"} err="failed to get container status \"603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b\": rpc error: code = NotFound desc = could not find container \"603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b\": container with ID starting with 603d1969910a5dae2d4dc18090d798941dfcbeff60737b0fd3cee4282667645b not found: ID does not exist"
Oct 31 04:06:21.751251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.751017    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"]
Oct 31 04:06:21.790380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:21.790351    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-833977-586fcfd6b9-5v79q"]
Oct 31 04:06:22.281366 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:22.281331    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:22.353825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:22.353791    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f715b5-f7d0-440c-9401-dc9cf82944e9" path="/var/lib/kubelet/pods/20f715b5-f7d0-440c-9401-dc9cf82944e9/volumes"
Oct 31 04:06:22.556131 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:22.556049    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:22.691946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:22.691908    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:23.191486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:23.191441    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:24.584779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.584750    5906 generic.go:356] "Generic (PLEG): container finished" podID="e2246c16-a828-4f23-9615-06911991656e" containerID="703014f2811c89030c1944d1070c7d8f67e2294891d815fa996344efd7b4ba64" exitCode=0
Oct 31 04:06:24.585263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.584789    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" event={"ID":"e2246c16-a828-4f23-9615-06911991656e","Type":"ContainerDied","Data":"703014f2811c89030c1944d1070c7d8f67e2294891d815fa996344efd7b4ba64"}
Oct 31 04:06:24.585263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.584842    5906 scope.go:122] "RemoveContainer" containerID="14f8cd699153e2d0c3c5dcebd0bc82aceac5d310faf73dc9c2f2225912e19313"
Oct 31 04:06:24.609981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.609962    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:06:24.633364 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633334    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633375    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633410    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633495    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633560    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633681 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633654    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633739 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633685    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:24.633739 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633704    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633771    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633798    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket\") pod \"e2246c16-a828-4f23-9615-06911991656e\" (UID: \"e2246c16-a828-4f23-9615-06911991656e\") "
Oct 31 04:06:24.633953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.633880    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:24.634174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634149    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:24.634297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634171    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:24.634297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634178    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:24.634511 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634492    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.634584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634512    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.634584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634524    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2246c16-a828-4f23-9615-06911991656e-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.634584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634535    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.634584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.634547    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.635879 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.635848    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:24.635879 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.635859    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:06:24.636129 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.636106    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "kube-api-access-vzwnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:24.636275 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.636241    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy" pod "e2246c16-a828-4f23-9615-06911991656e" (UID: "e2246c16-a828-4f23-9615-06911991656e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:24.735557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.735525    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2246c16-a828-4f23-9615-06911991656e-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.735557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.735555    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.735557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.735565    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzwnt\" (UniqueName: \"kubernetes.io/projected/e2246c16-a828-4f23-9615-06911991656e-kube-api-access-vzwnt\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.735806 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.735576    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2246c16-a828-4f23-9615-06911991656e-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:24.986539 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:24.986452    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:25.593793 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:25.593746    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"
Oct 31 04:06:25.593793 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:25.593784    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts" event={"ID":"e2246c16-a828-4f23-9615-06911991656e","Type":"ContainerDied","Data":"0efeef1292615fc8cc2641fd2b98ca8fd8fcec38292fd120a8cc025dee468609"}
Oct 31 04:06:25.594525 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:25.593849    5906 scope.go:122] "RemoveContainer" containerID="703014f2811c89030c1944d1070c7d8f67e2294891d815fa996344efd7b4ba64"
Oct 31 04:06:25.835818 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:25.835785    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"]
Oct 31 04:06:25.839968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:25.839944    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-f9a7fa-openshift-default-5849cd5cc5-snrts"]
Oct 31 04:06:26.353439 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:26.353397    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2246c16-a828-4f23-9615-06911991656e" path="/var/lib/kubelet/pods/e2246c16-a828-4f23-9615-06911991656e/volumes"
Oct 31 04:06:26.724831 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:26.724746    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:27.455681 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:27.455626    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:29.607299 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:29.607257    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:29.763492 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:29.763459    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:29.976449 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:29.976352    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"]
Oct 31 04:06:29.976639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:29.976584    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy" containerID="cri-o://92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e" gracePeriod=30
Oct 31 04:06:30.010653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:30.010602    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:30.445371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:30.445338    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:32.226683 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:32.226647    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:33.250095 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:33.250052    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:33.394983 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:33.394942    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:35.270643 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.270614    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:06:35.339280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339241    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339328    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339418    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339471    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339646 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339504    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339646 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339586    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339775 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339667    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339775 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339709    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339775 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339741    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token\") pod \"4b2a7874-12f4-49f8-8460-2932d97bf620\" (UID: \"4b2a7874-12f4-49f8-8460-2932d97bf620\") "
Oct 31 04:06:35.339942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339782    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:35.339942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339797    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:35.339942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339851    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:35.340110 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339972    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:35.340110 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.339988    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:35.340432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.340409    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.340432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.340429    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b2a7874-12f4-49f8-8460-2932d97bf620-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.340432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.340441    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.340686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.340457    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.340686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.340468    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.341815 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.341788    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:06:35.342164 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.342142    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "kube-api-access-hszjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:35.342411 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.342377    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:35.342411 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.342398    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token" pod "4b2a7874-12f4-49f8-8460-2932d97bf620" (UID: "4b2a7874-12f4-49f8-8460-2932d97bf620"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:35.441606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.441565    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.441606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.441596    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.441606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.441608    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b2a7874-12f4-49f8-8460-2932d97bf620-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.441606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.441617    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hszjn\" (UniqueName: \"kubernetes.io/projected/4b2a7874-12f4-49f8-8460-2932d97bf620-kube-api-access-hszjn\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:35.516299 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.516171    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:35.680907 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.680872    5906 generic.go:356] "Generic (PLEG): container finished" podID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerID="92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e" exitCode=0
Oct 31 04:06:35.681097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.680925    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"
Oct 31 04:06:35.681097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.680962    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" event={"ID":"4b2a7874-12f4-49f8-8460-2932d97bf620","Type":"ContainerDied","Data":"92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e"}
Oct 31 04:06:35.681097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.681019    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd" event={"ID":"4b2a7874-12f4-49f8-8460-2932d97bf620","Type":"ContainerDied","Data":"c7e83ed4ea6d8b8ecf263e4494a34736ede618a170fa82764421036de2aa2a75"}
Oct 31 04:06:35.681097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.681046    5906 scope.go:122] "RemoveContainer" containerID="92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e"
Oct 31 04:06:35.692599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.692573    5906 scope.go:122] "RemoveContainer" containerID="9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"
Oct 31 04:06:35.702694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.702673    5906 scope.go:122] "RemoveContainer" containerID="92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e"
Oct 31 04:06:35.702972 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:06:35.702947    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e\": container with ID starting with 92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e not found: ID does not exist" containerID="92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e"
Oct 31 04:06:35.703052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.702985    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e"} err="failed to get container status \"92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e\": rpc error: code = NotFound desc = could not find container \"92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e\": container with ID starting with 92188a7815dd4a89e28f6fbc42569a926fe0849c29cdcd0004fe3e386c318e7e not found: ID does not exist"
Oct 31 04:06:35.703052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.703014    5906 scope.go:122] "RemoveContainer" containerID="9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"
Oct 31 04:06:35.703306 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:06:35.703287    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb\": container with ID starting with 9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb not found: ID does not exist" containerID="9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"
Oct 31 04:06:35.703353 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.703313    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb"} err="failed to get container status \"9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb\": rpc error: code = NotFound desc = could not find container \"9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb\": container with ID starting with 9000745a363ead385f735fa3f8674890096dda6b072b2236191970306220efbb not found: ID does not exist"
Oct 31 04:06:35.852677 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.852579    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"]
Oct 31 04:06:35.855550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:35.855518    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-833977-openshift-default-6c66d5df9c-cvcbd"]
Oct 31 04:06:36.352994 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:36.352960    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" path="/var/lib/kubelet/pods/4b2a7874-12f4-49f8-8460-2932d97bf620/volumes"
Oct 31 04:06:36.382457 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:06:36.382419    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:06:36.995613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:36.995573    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:41.510057 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:41.510022    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:41.657684 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:41.657645    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:42.883130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:42.883096    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"]
Oct 31 04:06:43.118555 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:43.118523    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:43.261579 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:43.261545    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:46.221425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:46.221392    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:47.883678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:47.883633    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerName="payload-pre-processing" containerID="cri-o://9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d" gracePeriod=25
Oct 31 04:06:51.091876 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:51.091750    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:52.566988 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:52.566948    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:52.867754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:52.867645    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:55.596807 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:55.596770    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:06:57.167049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:57.167000    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:06:59.004363 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:59.004326    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:06:59.004790 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:59.004449    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:06:59.250903 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:06:59.250867    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:00.566801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.566766    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:00.906729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.906636    5906 generic.go:356] "Generic (PLEG): container finished" podID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerID="ea07630487e90ba96ba551eb33d94857af930af75b8ae58fb4ae59fd213b30b0" exitCode=0
Oct 31 04:07:00.906729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.906676    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" event={"ID":"5e688e64-20e8-471d-97b5-55ebadcaa984","Type":"ContainerDied","Data":"ea07630487e90ba96ba551eb33d94857af930af75b8ae58fb4ae59fd213b30b0"}
Oct 31 04:07:00.907198 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.907175    5906 scope.go:122] "RemoveContainer" containerID="ea07630487e90ba96ba551eb33d94857af930af75b8ae58fb4ae59fd213b30b0"
Oct 31 04:07:00.909483 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.909448    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:07:00.909595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.909522    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:07:00.909595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:00.909553    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:07:01.260141 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:01.260099    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:01.915333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:01.915295    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" event={"ID":"5e688e64-20e8-471d-97b5-55ebadcaa984","Type":"ContainerStarted","Data":"d38bd453e1f723d2c1b1b540c6f8539d198b1a8065ef990a8fe7b8722e45dfaf"}
Oct 31 04:07:02.045918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:02.045881    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:07:02.046338 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:02.046200    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 31 04:07:02.046541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:02.046518    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 31 04:07:02.206481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:02.206345    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:02.376435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:02.376171    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:02.853375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:02.853315    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:03.040758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:03.040717    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 31 04:07:03.041330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:03.040796    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 31 04:07:04.040662 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:04.040626    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 31 04:07:04.040849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:04.040695    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 31 04:07:04.232541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:04.232499    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"]
Oct 31 04:07:04.377881 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:04.376562    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:05.040647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:05.040602    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 31 04:07:05.040875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:05.040678    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 31 04:07:05.691870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:05.691827    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:06.057381 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:06.057337    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 31 04:07:06.058011 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:06.057973    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 31 04:07:06.396621 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:06.396529    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:07:07.058416 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:07.058363    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused" start-of-body=
Oct 31 04:07:07.066367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:07.066327    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.130:15021/healthz/ready\": dial tcp 10.134.0.130:15021: connect: connection refused"
Oct 31 04:07:07.165143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:07.165109    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:07.168414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:07.168381    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:07:08.044935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.044899    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:07:08.045325 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.045300    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:07:08.046166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.046144    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:07:08.160244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.160167    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 04:07:08.256721 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.256687    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2\" (UniqueName: \"kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2\") pod \"1e944735-7b85-4160-a3ec-a4cefd28f746\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") "
Oct 31 04:07:08.256937 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.256860    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume\") pod \"1e944735-7b85-4160-a3ec-a4cefd28f746\" (UID: \"1e944735-7b85-4160-a3ec-a4cefd28f746\") "
Oct 31 04:07:08.257275 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.257251    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume" pod "1e944735-7b85-4160-a3ec-a4cefd28f746" (UID: "1e944735-7b85-4160-a3ec-a4cefd28f746"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:07:08.258932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.258910    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2" pod "1e944735-7b85-4160-a3ec-a4cefd28f746" (UID: "1e944735-7b85-4160-a3ec-a4cefd28f746"). InnerVolumeSpecName "kube-api-access-r6vk2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:08.359676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.359611    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e944735-7b85-4160-a3ec-a4cefd28f746-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:08.359676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.359648    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6vk2\" (UniqueName: \"kubernetes.io/projected/1e944735-7b85-4160-a3ec-a4cefd28f746-kube-api-access-r6vk2\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:08.991081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.991028    5906 generic.go:356] "Generic (PLEG): container finished" podID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerID="9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d" exitCode=0
Oct 31 04:07:08.991323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.991086    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"
Oct 31 04:07:08.991323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.991085    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" event={"ID":"1e944735-7b85-4160-a3ec-a4cefd28f746","Type":"ContainerDied","Data":"9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d"}
Oct 31 04:07:08.991323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.991158    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l" event={"ID":"1e944735-7b85-4160-a3ec-a4cefd28f746","Type":"ContainerDied","Data":"ed7419e6427bd02f19c56893274954cae2d620d71195bf0708b373a79f60f112"}
Oct 31 04:07:08.991323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:08.991189    5906 scope.go:122] "RemoveContainer" containerID="9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d"
Oct 31 04:07:09.000364 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:09.000330    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:09.003732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:09.003555    5906 scope.go:122] "RemoveContainer" containerID="9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d"
Oct 31 04:07:09.003849 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:09.003830    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d\": container with ID starting with 9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d not found: ID does not exist" containerID="9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d"
Oct 31 04:07:09.003899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:09.003859    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d"} err="failed to get container status \"9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d\": rpc error: code = NotFound desc = could not find container \"9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d\": container with ID starting with 9f54a1ad5efa3b913bfcddec59bc058c5a7c751fd7ccd0679f1c29ef02d91b4d not found: ID does not exist"
Oct 31 04:07:09.164391 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:09.164355    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"]
Oct 31 04:07:09.169839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:09.169802    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-537e34-7fb977cc87-r4r4l"]
Oct 31 04:07:10.194984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:10.194950    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:10.353812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:10.353779    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" path="/var/lib/kubelet/pods/1e944735-7b85-4160-a3ec-a4cefd28f746/volumes"
Oct 31 04:07:11.488750 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:11.488709    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"]
Oct 31 04:07:11.731784 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:11.731700    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:11.871802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:11.871766    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:12.152592 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:12.152562    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:12.292770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:12.292706    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:14.110931 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:14.110791    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:14.320157 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:14.320125    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:14.705709 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:14.705675    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:15.592935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:15.592903    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:15.735707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:15.735653    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:16.489895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:16.489813    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerName="payload-pre-processing" containerID="cri-o://3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2" gracePeriod=25
Oct 31 04:07:17.333875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:17.333836    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:07:18.008304 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:18.008265    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:19.292404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.292360    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main" containerID="cri-o://0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1" gracePeriod=45
Oct 31 04:07:19.504956 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.504920    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:19.631897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.631871    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 04:07:19.797678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.797636    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home\") pod \"169384ae-a553-458b-b56b-7c311ab17d38\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") "
Oct 31 04:07:19.797914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.797736    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r\" (UniqueName: \"kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r\") pod \"169384ae-a553-458b-b56b-7c311ab17d38\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") "
Oct 31 04:07:19.797914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.797834    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs\" (UniqueName: \"kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs\") pod \"169384ae-a553-458b-b56b-7c311ab17d38\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") "
Oct 31 04:07:19.798057 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.797985    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home" pod "169384ae-a553-458b-b56b-7c311ab17d38" (UID: "169384ae-a553-458b-b56b-7c311ab17d38"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.798382 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.798356    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache\") pod \"169384ae-a553-458b-b56b-7c311ab17d38\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") "
Oct 31 04:07:19.798817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.798734    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache" pod "169384ae-a553-458b-b56b-7c311ab17d38" (UID: "169384ae-a553-458b-b56b-7c311ab17d38"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.798963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.798868    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir\") pod \"169384ae-a553-458b-b56b-7c311ab17d38\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") "
Oct 31 04:07:19.799053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.798958    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm\") pod \"169384ae-a553-458b-b56b-7c311ab17d38\" (UID: \"169384ae-a553-458b-b56b-7c311ab17d38\") "
Oct 31 04:07:19.799147 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.799122    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir" pod "169384ae-a553-458b-b56b-7c311ab17d38" (UID: "169384ae-a553-458b-b56b-7c311ab17d38"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.799522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.799498    5906 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-home\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.799611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.799522    5906 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-model-cache\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.799611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.799560    5906 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-tmp-dir\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.800403 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.800380    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r" pod "169384ae-a553-458b-b56b-7c311ab17d38" (UID: "169384ae-a553-458b-b56b-7c311ab17d38"). InnerVolumeSpecName "kube-api-access-6hd6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:19.800541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.800525    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs" pod "169384ae-a553-458b-b56b-7c311ab17d38" (UID: "169384ae-a553-458b-b56b-7c311ab17d38"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:07:19.801416 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.801398    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm" pod "169384ae-a553-458b-b56b-7c311ab17d38" (UID: "169384ae-a553-458b-b56b-7c311ab17d38"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.900981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.900886    5906 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/169384ae-a553-458b-b56b-7c311ab17d38-dshm\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.900981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.900921    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hd6r\" (UniqueName: \"kubernetes.io/projected/169384ae-a553-458b-b56b-7c311ab17d38-kube-api-access-6hd6r\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.900981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:19.900935    5906 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/169384ae-a553-458b-b56b-7c311ab17d38-tls-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:20.109986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.109953    5906 generic.go:356] "Generic (PLEG): container finished" podID="169384ae-a553-458b-b56b-7c311ab17d38" containerID="0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1" exitCode=0
Oct 31 04:07:20.110148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.110004    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"
Oct 31 04:07:20.110148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.110039    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" event={"ID":"169384ae-a553-458b-b56b-7c311ab17d38","Type":"ContainerDied","Data":"0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1"}
Oct 31 04:07:20.110148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.110085    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd" event={"ID":"169384ae-a553-458b-b56b-7c311ab17d38","Type":"ContainerDied","Data":"710f912d281d777023eab2b537992bfac64a173fdac10eae2122438c4f66c1a6"}
Oct 31 04:07:20.110148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.110107    5906 scope.go:122] "RemoveContainer" containerID="0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1"
Oct 31 04:07:20.121005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.120986    5906 scope.go:122] "RemoveContainer" containerID="0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1"
Oct 31 04:07:20.121309 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:20.121287    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1\": container with ID starting with 0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1 not found: ID does not exist" containerID="0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1"
Oct 31 04:07:20.121394 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.121318    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1"} err="failed to get container status \"0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1\": rpc error: code = NotFound desc = could not find container \"0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1\": container with ID starting with 0afb2f96dda5c9a6470907c74e0aeb53a111587f2faea384ce22dda4cf8266d1 not found: ID does not exist"
Oct 31 04:07:20.280621 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.280585    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"]
Oct 31 04:07:20.284698 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.284670    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-a64fd2/distinct-w2-a64fd2-kserve-6f6f65b7f6-zskmd"]
Oct 31 04:07:20.353812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.353768    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="169384ae-a553-458b-b56b-7c311ab17d38" path="/var/lib/kubelet/pods/169384ae-a553-458b-b56b-7c311ab17d38/volumes"
Oct 31 04:07:20.887108 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:20.887068    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:21.317494 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:21.317456    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:24.458716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:24.458679    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:27.045595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:27.045557    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:27.334256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:27.334123    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:07:29.185893 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:29.185858    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:32.251947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:32.251915    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"]
Oct 31 04:07:32.354823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:32.354793    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"]
Oct 31 04:07:32.587503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:32.587427    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:32.731828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:32.731794    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:33.623016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:33.622971    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:33.771720 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:33.771684    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:34.799916 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:34.799880    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:36.382693 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:36.382650    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:07:36.566442 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.566401    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:36.732298 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.732264    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:36.777817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.777789    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 04:07:36.792333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.792304    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume\") pod \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") "
Oct 31 04:07:36.792510 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.792494    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk\" (UniqueName: \"kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk\") pod \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\" (UID: \"c7b2fc3a-dc93-437c-8cfc-f7727595709c\") "
Oct 31 04:07:36.792672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.792650    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume" pod "c7b2fc3a-dc93-437c-8cfc-f7727595709c" (UID: "c7b2fc3a-dc93-437c-8cfc-f7727595709c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:07:36.792959 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.792941    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7b2fc3a-dc93-437c-8cfc-f7727595709c-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:36.794647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.794627    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk" pod "c7b2fc3a-dc93-437c-8cfc-f7727595709c" (UID: "c7b2fc3a-dc93-437c-8cfc-f7727595709c"). InnerVolumeSpecName "kube-api-access-vzrxk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:36.894259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:36.894223    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzrxk\" (UniqueName: \"kubernetes.io/projected/c7b2fc3a-dc93-437c-8cfc-f7727595709c-kube-api-access-vzrxk\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:37.233831 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.233794    5906 generic.go:356] "Generic (PLEG): container finished" podID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerID="3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2" exitCode=0
Oct 31 04:07:37.234016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.233846    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"
Oct 31 04:07:37.234016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.233882    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" event={"ID":"c7b2fc3a-dc93-437c-8cfc-f7727595709c","Type":"ContainerDied","Data":"3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2"}
Oct 31 04:07:37.234016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.233940    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn" event={"ID":"c7b2fc3a-dc93-437c-8cfc-f7727595709c","Type":"ContainerDied","Data":"b7a2c9719d2bbc5cf85dbdddcf0c07e75d00a7de0d548267262c79de3b4b7010"}
Oct 31 04:07:37.234016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.233962    5906 scope.go:122] "RemoveContainer" containerID="3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2"
Oct 31 04:07:37.244153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.244132    5906 scope.go:122] "RemoveContainer" containerID="3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2"
Oct 31 04:07:37.244463 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:37.244440    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2\": container with ID starting with 3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2 not found: ID does not exist" containerID="3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2"
Oct 31 04:07:37.244544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.244475    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2"} err="failed to get container status \"3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2\": rpc error: code = NotFound desc = could not find container \"3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2\": container with ID starting with 3403731e3b8674fe07a34c7520b44cb3040cdb364b6f1961f569d3f37adfd8d2 not found: ID does not exist"
Oct 31 04:07:37.253143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.253111    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr" podUID="48c9ffad-9df1-429a-b483-170065a30abb" containerName="payload-processing" containerID="cri-o://879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e" gracePeriod=25
Oct 31 04:07:37.355781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.355736    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt" podUID="1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" containerName="payload-pre-processing" containerID="cri-o://83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3" gracePeriod=25
Oct 31 04:07:37.362153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.362119    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:37.407174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.407141    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"]
Oct 31 04:07:37.462645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.461616    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-a64fd2-798fbc8d79-htcrn"]
Oct 31 04:07:37.534770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.534738    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:07:37.602028 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.601994    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume\") pod \"48c9ffad-9df1-429a-b483-170065a30abb\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") "
Oct 31 04:07:37.602214 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.602143    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v\" (UniqueName: \"kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v\") pod \"48c9ffad-9df1-429a-b483-170065a30abb\" (UID: \"48c9ffad-9df1-429a-b483-170065a30abb\") "
Oct 31 04:07:37.602506 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.602470    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume" pod "48c9ffad-9df1-429a-b483-170065a30abb" (UID: "48c9ffad-9df1-429a-b483-170065a30abb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:07:37.602716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.602688    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48c9ffad-9df1-429a-b483-170065a30abb-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:37.605110 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.605086    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v" pod "48c9ffad-9df1-429a-b483-170065a30abb" (UID: "48c9ffad-9df1-429a-b483-170065a30abb"). InnerVolumeSpecName "kube-api-access-npz2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:37.655335 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.655309    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:07:37.703509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.703471    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf\" (UniqueName: \"kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf\") pod \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") "
Oct 31 04:07:37.703731 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.703713    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume\") pod \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\" (UID: \"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69\") "
Oct 31 04:07:37.704249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.704175    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume" pod "1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" (UID: "1828d0f0-cdbc-4f98-a047-a6ecb71d0a69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:07:37.704249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.704234    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npz2v\" (UniqueName: \"kubernetes.io/projected/48c9ffad-9df1-429a-b483-170065a30abb-kube-api-access-npz2v\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:37.705639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.705613    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf" pod "1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" (UID: "1828d0f0-cdbc-4f98-a047-a6ecb71d0a69"). InnerVolumeSpecName "kube-api-access-zcwkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:37.805997 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.805921    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:37.805997 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:37.805962    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcwkf\" (UniqueName: \"kubernetes.io/projected/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69-kube-api-access-zcwkf\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:38.242507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.242465    5906 generic.go:356] "Generic (PLEG): container finished" podID="1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" containerID="83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3" exitCode=0
Oct 31 04:07:38.242507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.242508    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"
Oct 31 04:07:38.242755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.242549    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt" event={"ID":"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69","Type":"ContainerDied","Data":"83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3"}
Oct 31 04:07:38.242755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.242617    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt" event={"ID":"1828d0f0-cdbc-4f98-a047-a6ecb71d0a69","Type":"ContainerDied","Data":"83dbcae9558b6d70cec3201af71011984c8e349903b4b87e4bd831c63a9bad88"}
Oct 31 04:07:38.242755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.242653    5906 scope.go:122] "RemoveContainer" containerID="83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3"
Oct 31 04:07:38.245761 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.245737    5906 generic.go:356] "Generic (PLEG): container finished" podID="48c9ffad-9df1-429a-b483-170065a30abb" containerID="879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e" exitCode=0
Oct 31 04:07:38.245892 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.245783    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr" event={"ID":"48c9ffad-9df1-429a-b483-170065a30abb","Type":"ContainerDied","Data":"879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e"}
Oct 31 04:07:38.245892 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.245797    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"
Oct 31 04:07:38.245892 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.245823    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr" event={"ID":"48c9ffad-9df1-429a-b483-170065a30abb","Type":"ContainerDied","Data":"26f553e6b146337688d5f703fcdf866bd31dad1679e1744eb6cb44ab9808adae"}
Oct 31 04:07:38.255568 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.255546    5906 scope.go:122] "RemoveContainer" containerID="83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3"
Oct 31 04:07:38.256009 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:38.255916    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3\": container with ID starting with 83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3 not found: ID does not exist" containerID="83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3"
Oct 31 04:07:38.256009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.255951    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3"} err="failed to get container status \"83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3\": rpc error: code = NotFound desc = could not find container \"83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3\": container with ID starting with 83fe3686f82623f87b13c347e8b568b80d789149f778ab54065e37be5d1a68a3 not found: ID does not exist"
Oct 31 04:07:38.256009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.255979    5906 scope.go:122] "RemoveContainer" containerID="879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e"
Oct 31 04:07:38.266636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.266615    5906 scope.go:122] "RemoveContainer" containerID="879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e"
Oct 31 04:07:38.266928 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:07:38.266907    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e\": container with ID starting with 879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e not found: ID does not exist" containerID="879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e"
Oct 31 04:07:38.266990 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.266941    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e"} err="failed to get container status \"879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e\": rpc error: code = NotFound desc = could not find container \"879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e\": container with ID starting with 879f37d99e7f9d5a596267f2169996cd52b1db73335755942625578992975c0e not found: ID does not exist"
Oct 31 04:07:38.352953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.352922    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" path="/var/lib/kubelet/pods/c7b2fc3a-dc93-437c-8cfc-f7727595709c/volumes"
Oct 31 04:07:38.413861 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.413827    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"]
Oct 31 04:07:38.465318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.465278    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-ada9ed-55ddccdc67-bd6zt"]
Oct 31 04:07:38.532710 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.532666    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"]
Oct 31 04:07:38.537869 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:38.537836    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-ada9ed-5d9b5c4795-275zr"]
Oct 31 04:07:39.927567 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:39.927522    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:40.362081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:40.362046    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" path="/var/lib/kubelet/pods/1828d0f0-cdbc-4f98-a047-a6ecb71d0a69/volumes"
Oct 31 04:07:40.362552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:40.362536    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48c9ffad-9df1-429a-b483-170065a30abb" path="/var/lib/kubelet/pods/48c9ffad-9df1-429a-b483-170065a30abb/volumes"
Oct 31 04:07:40.631345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:40.631257    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:40.773677 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:40.773643    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:42.442552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:42.442519    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:44.736472 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:44.736433    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:48.143278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:48.143238    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:49.262463 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:49.262429    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:53.165000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:53.164942    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:54.634115 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:54.634078    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:54.834543 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:54.834503    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:54.987413 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:54.987269    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:55.312333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:55.312252    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:55.638801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:55.638687    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:55.783191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:55.783162    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:07:59.748883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:07:59.748835    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:00.291086 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:00.290978    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:02.141751 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:02.141715    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:02.296309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:02.296232    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:02.721305 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:02.721267    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:02.864838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:02.864802    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:03.055662 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:03.055630    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:04.305411 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:04.305373    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:06.382042 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:06.382008    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:08:08.810476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:08.810421    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:13.330948 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:13.330895    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:17.093977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:17.093938    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:17.239429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:17.239395    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:17.793686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:17.793648    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:21.104493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.104407    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:21.242053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.242020    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:21.571808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.571773    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"]
Oct 31 04:08:21.572112 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.572088    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" podUID="521f908c-8733-4b71-af51-4f0fcd339c09" containerName="maas-api" containerID="cri-o://ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7" gracePeriod=30
Oct 31 04:08:21.665836 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.665806    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:21.901405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.901377    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 04:08:21.995563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.995531    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\") pod \"521f908c-8733-4b71-af51-4f0fcd339c09\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") "
Oct 31 04:08:21.995788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.995648    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898\" (UniqueName: \"kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898\") pod \"521f908c-8733-4b71-af51-4f0fcd339c09\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") "
Oct 31 04:08:21.995788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.995726    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs\") pod \"521f908c-8733-4b71-af51-4f0fcd339c09\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") "
Oct 31 04:08:21.995932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.995832    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca\") pod \"521f908c-8733-4b71-af51-4f0fcd339c09\" (UID: \"521f908c-8733-4b71-af51-4f0fcd339c09\") "
Oct 31 04:08:21.996289 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.996256    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca" pod "521f908c-8733-4b71-af51-4f0fcd339c09" (UID: "521f908c-8733-4b71-af51-4f0fcd339c09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:21.996733 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.996713    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/521f908c-8733-4b71-af51-4f0fcd339c09-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:21.997860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.997829    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls" pod "521f908c-8733-4b71-af51-4f0fcd339c09" (UID: "521f908c-8733-4b71-af51-4f0fcd339c09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:08:21.998236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.998176    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898" pod "521f908c-8733-4b71-af51-4f0fcd339c09" (UID: "521f908c-8733-4b71-af51-4f0fcd339c09"). InnerVolumeSpecName "kube-api-access-hx898". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:21.998236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:21.998220    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs" pod "521f908c-8733-4b71-af51-4f0fcd339c09" (UID: "521f908c-8733-4b71-af51-4f0fcd339c09"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:08:22.097905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.097874    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:22.097905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.097902    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/521f908c-8733-4b71-af51-4f0fcd339c09-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:22.097905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.097912    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx898\" (UniqueName: \"kubernetes.io/projected/521f908c-8733-4b71-af51-4f0fcd339c09-kube-api-access-hx898\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:22.608865 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.608823    5906 generic.go:356] "Generic (PLEG): container finished" podID="521f908c-8733-4b71-af51-4f0fcd339c09" containerID="ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7" exitCode=0
Oct 31 04:08:22.609399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.608902    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"
Oct 31 04:08:22.609399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.608913    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" event={"ID":"521f908c-8733-4b71-af51-4f0fcd339c09","Type":"ContainerDied","Data":"ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7"}
Oct 31 04:08:22.609399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.608972    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs" event={"ID":"521f908c-8733-4b71-af51-4f0fcd339c09","Type":"ContainerDied","Data":"619f35e361b9aa242c069d63be92fcfe8eb5915e02f4fc15d310e678232c5eb4"}
Oct 31 04:08:22.609399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.609001    5906 scope.go:122] "RemoveContainer" containerID="ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7"
Oct 31 04:08:22.619860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.619827    5906 scope.go:122] "RemoveContainer" containerID="ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7"
Oct 31 04:08:22.620144 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:22.620117    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7\": container with ID starting with ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7 not found: ID does not exist" containerID="ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7"
Oct 31 04:08:22.620256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.620154    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7"} err="failed to get container status \"ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7\": rpc error: code = NotFound desc = could not find container \"ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7\": container with ID starting with ce2f462a26a78812f76692a59d7a2bcbb7ca4a8b54999ab212fd5a87e9fdd8a7 not found: ID does not exist"
Oct 31 04:08:22.641300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.641259    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"]
Oct 31 04:08:22.641653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.641620    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" podUID="a79b0003-c031-4b13-9375-97a1d0ccefbb" containerName="maas-api" containerID="cri-o://1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118" gracePeriod=30
Oct 31 04:08:22.736668 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.736630    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:22.929371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.929280    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"]
Oct 31 04:08:22.933221 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.933174    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-537e34-5c757d45fc-hqkvs"]
Oct 31 04:08:22.997099 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:22.997068    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:08:23.023346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.023316    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs\") pod \"a79b0003-c031-4b13-9375-97a1d0ccefbb\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") "
Oct 31 04:08:23.023545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.023419    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\") pod \"a79b0003-c031-4b13-9375-97a1d0ccefbb\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") "
Oct 31 04:08:23.023545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.023538    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np\" (UniqueName: \"kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np\") pod \"a79b0003-c031-4b13-9375-97a1d0ccefbb\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") "
Oct 31 04:08:23.023700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.023686    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca\") pod \"a79b0003-c031-4b13-9375-97a1d0ccefbb\" (UID: \"a79b0003-c031-4b13-9375-97a1d0ccefbb\") "
Oct 31 04:08:23.024036 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.024008    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca" pod "a79b0003-c031-4b13-9375-97a1d0ccefbb" (UID: "a79b0003-c031-4b13-9375-97a1d0ccefbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:23.024331 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.024314    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a79b0003-c031-4b13-9375-97a1d0ccefbb-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:23.025439 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.025414    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs" pod "a79b0003-c031-4b13-9375-97a1d0ccefbb" (UID: "a79b0003-c031-4b13-9375-97a1d0ccefbb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:08:23.025949 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.025930    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls" pod "a79b0003-c031-4b13-9375-97a1d0ccefbb" (UID: "a79b0003-c031-4b13-9375-97a1d0ccefbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:08:23.026029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.025961    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np" pod "a79b0003-c031-4b13-9375-97a1d0ccefbb" (UID: "a79b0003-c031-4b13-9375-97a1d0ccefbb"). InnerVolumeSpecName "kube-api-access-gq4np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:23.067622 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.067585    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:23.132589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.132504    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:23.132589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.132542    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a79b0003-c031-4b13-9375-97a1d0ccefbb-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:23.132589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.132557    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq4np\" (UniqueName: \"kubernetes.io/projected/a79b0003-c031-4b13-9375-97a1d0ccefbb-kube-api-access-gq4np\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:23.339589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.339553    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:23.632730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.632640    5906 generic.go:356] "Generic (PLEG): container finished" podID="a79b0003-c031-4b13-9375-97a1d0ccefbb" containerID="1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118" exitCode=0
Oct 31 04:08:23.632730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.632693    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" event={"ID":"a79b0003-c031-4b13-9375-97a1d0ccefbb","Type":"ContainerDied","Data":"1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118"}
Oct 31 04:08:23.632730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.632728    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t" event={"ID":"a79b0003-c031-4b13-9375-97a1d0ccefbb","Type":"ContainerDied","Data":"f836c3c405ce1e4e059be2c9ef258eb2b0cb3480b716a4334d214ed9801b58b2"}
Oct 31 04:08:23.633180 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.632750    5906 scope.go:122] "RemoveContainer" containerID="1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118"
Oct 31 04:08:23.633180 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.632831    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"
Oct 31 04:08:23.646571 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.646541    5906 scope.go:122] "RemoveContainer" containerID="1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118"
Oct 31 04:08:23.646893 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:23.646870    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118\": container with ID starting with 1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118 not found: ID does not exist" containerID="1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118"
Oct 31 04:08:23.646960 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.646902    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118"} err="failed to get container status \"1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118\": rpc error: code = NotFound desc = could not find container \"1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118\": container with ID starting with 1a7e9e4e82f71d4cffe62896a064d31b6f336490ff536333285a19e2f9e92118 not found: ID does not exist"
Oct 31 04:08:23.804314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.804268    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"]
Oct 31 04:08:23.856248 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.856201    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ada9ed-b7d4dd94c-qn67t"]
Oct 31 04:08:23.983506 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:23.983400    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:24.195245 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:24.195191    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:24.353596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:24.353559    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="521f908c-8733-4b71-af51-4f0fcd339c09" path="/var/lib/kubelet/pods/521f908c-8733-4b71-af51-4f0fcd339c09/volumes"
Oct 31 04:08:24.354151 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:24.354127    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a79b0003-c031-4b13-9375-97a1d0ccefbb" path="/var/lib/kubelet/pods/a79b0003-c031-4b13-9375-97a1d0ccefbb/volumes"
Oct 31 04:08:24.434570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:24.434537    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:24.576414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:24.576376    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:25.130596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:25.130558    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:27.402259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:27.402224    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:29.181087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:29.181055    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:30.098427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:30.098391    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:30.238040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:30.238002    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:30.443496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:30.443391    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:31.682407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:31.682371    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:31.832946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:31.832898    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:32.027768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:32.027728    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:32.032261 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:32.032230    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"]
Oct 31 04:08:32.032514 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:32.032487    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy" containerID="cri-o://92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889" gracePeriod=30
Oct 31 04:08:34.073447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:34.073406    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:36.382367 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:36.382325    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:08:37.299602 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.299533    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:08:37.454899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.454858    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.454912    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.454974    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455078    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455127    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455177    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455186    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:37.455477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455364    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455508    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455550    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs\") pod \"d2691af5-1c21-407e-9627-0ebc83d05b86\" (UID: \"d2691af5-1c21-407e-9627-0ebc83d05b86\") "
Oct 31 04:08:37.455898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455614    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:37.455898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455702    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:37.455898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455793    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:37.456186 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.455893    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:37.456688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.456511    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.456688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.456539    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.456688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.456588    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.456688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.456605    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2691af5-1c21-407e-9627-0ebc83d05b86-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.456688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.456620    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.457536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.457513    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:08:37.457651 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.457602    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "kube-api-access-ds58d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:37.457717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.457702    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:37.457852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.457834    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token" pod "d2691af5-1c21-407e-9627-0ebc83d05b86" (UID: "d2691af5-1c21-407e-9627-0ebc83d05b86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:37.558345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.558247    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.558345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.558282    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds58d\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-kube-api-access-ds58d\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.558345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.558293    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.558345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.558302    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2691af5-1c21-407e-9627-0ebc83d05b86-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:37.738640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.738606    5906 generic.go:356] "Generic (PLEG): container finished" podID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerID="92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889" exitCode=0
Oct 31 04:08:37.738833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.738660    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"
Oct 31 04:08:37.738833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.738691    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" event={"ID":"d2691af5-1c21-407e-9627-0ebc83d05b86","Type":"ContainerDied","Data":"92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889"}
Oct 31 04:08:37.738833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.738749    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr" event={"ID":"d2691af5-1c21-407e-9627-0ebc83d05b86","Type":"ContainerDied","Data":"843b99f2aa8af1a9beef0684b6858559faf51056d7ec74af50ed76da7f6cc95b"}
Oct 31 04:08:37.738833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.738774    5906 scope.go:122] "RemoveContainer" containerID="92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889"
Oct 31 04:08:37.754193 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.754161    5906 scope.go:122] "RemoveContainer" containerID="d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"
Oct 31 04:08:37.764366 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.764345    5906 scope.go:122] "RemoveContainer" containerID="92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889"
Oct 31 04:08:37.764703 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:37.764680    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889\": container with ID starting with 92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889 not found: ID does not exist" containerID="92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889"
Oct 31 04:08:37.764758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.764718    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889"} err="failed to get container status \"92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889\": rpc error: code = NotFound desc = could not find container \"92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889\": container with ID starting with 92681fc1a618f557f00177b4300848e4a829078c1710f6c44a691f342e027889 not found: ID does not exist"
Oct 31 04:08:37.764758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.764737    5906 scope.go:122] "RemoveContainer" containerID="d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"
Oct 31 04:08:37.765027 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:37.765004    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d\": container with ID starting with d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d not found: ID does not exist" containerID="d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"
Oct 31 04:08:37.765074 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.765034    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d"} err="failed to get container status \"d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d\": rpc error: code = NotFound desc = could not find container \"d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d\": container with ID starting with d0917613218326e96a9c23950fb5f7840a1e7b9373fe394f9ff934d4fc2a0f3d not found: ID does not exist"
Oct 31 04:08:37.909806 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.909725    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"]
Oct 31 04:08:37.961517 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:37.961453    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ada9ed-openshift-default-6bb6b5dcd5-4xmjr"]
Oct 31 04:08:38.352809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:38.352775    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" path="/var/lib/kubelet/pods/d2691af5-1c21-407e-9627-0ebc83d05b86/volumes"
Oct 31 04:08:38.871796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:38.871762    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:42.723586 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:42.723540    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"]
Oct 31 04:08:42.724094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:42.723805    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" podUID="7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" containerName="maas-api" containerID="cri-o://1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5" gracePeriod=30
Oct 31 04:08:42.817710 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:42.817667    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:43.035535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.035508    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 04:08:43.152536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.152494    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs\") pod \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") "
Oct 31 04:08:43.152717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.152608    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s\" (UniqueName: \"kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s\") pod \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") "
Oct 31 04:08:43.152809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.152724    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\") pod \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") "
Oct 31 04:08:43.152809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.152757    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca\") pod \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\" (UID: \"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb\") "
Oct 31 04:08:43.153274 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.153197    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca" pod "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" (UID: "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:43.155184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.155152    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s" pod "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" (UID: "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb"). InnerVolumeSpecName "kube-api-access-zfm9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:43.155382 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.155267    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls" pod "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" (UID: "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:08:43.155382 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.155367    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs" pod "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" (UID: "7a41ca42-cd82-4aaa-b6f7-33c2df136fdb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:08:43.255443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.255408    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:43.255443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.255442    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:43.255443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.255452    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:43.255690 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.255461    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfm9s\" (UniqueName: \"kubernetes.io/projected/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb-kube-api-access-zfm9s\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:43.781694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.781660    5906 generic.go:356] "Generic (PLEG): container finished" podID="7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" containerID="1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5" exitCode=0
Oct 31 04:08:43.782153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.781704    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"
Oct 31 04:08:43.782153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.781749    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" event={"ID":"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb","Type":"ContainerDied","Data":"1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5"}
Oct 31 04:08:43.782153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.781801    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm" event={"ID":"7a41ca42-cd82-4aaa-b6f7-33c2df136fdb","Type":"ContainerDied","Data":"026a96fc31b4bcf9d693b8ac7b853c0a19beb651653a4262a2cdf2e26c0a8c64"}
Oct 31 04:08:43.782153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.781825    5906 scope.go:122] "RemoveContainer" containerID="1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5"
Oct 31 04:08:43.793019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.793002    5906 scope.go:122] "RemoveContainer" containerID="1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5"
Oct 31 04:08:43.793330 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:08:43.793308    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5\": container with ID starting with 1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5 not found: ID does not exist" containerID="1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5"
Oct 31 04:08:43.793415 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.793340    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5"} err="failed to get container status \"1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5\": rpc error: code = NotFound desc = could not find container \"1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5\": container with ID starting with 1599f1524087994391fa0bbd1d422dd893e77b52c6edf8b155cc99602607a5f5 not found: ID does not exist"
Oct 31 04:08:43.952006 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:43.951966    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"]
Oct 31 04:08:44.003802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:44.003765    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-a64fd2-649576bc9b-dc2hm"]
Oct 31 04:08:44.277555 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:44.277521    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:44.353979 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:44.353941    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" path="/var/lib/kubelet/pods/7a41ca42-cd82-4aaa-b6f7-33c2df136fdb/volumes"
Oct 31 04:08:44.497579 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:44.497537    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:44.651794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:44.648084    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:44.793259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:44.792786    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:45.204795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:45.204761    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:49.292671 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:49.292629    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:51.301407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:51.301346    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:51.445979 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:51.445944    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:51.658021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:51.657931    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:53.807120 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:53.807083    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:56.575559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:56.575523    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"]
Oct 31 04:08:56.677559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:56.677525    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"]
Oct 31 04:08:56.920888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:56.920806    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:57.126810 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:57.126775    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:08:58.239179 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:08:58.239138    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:01.576570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.576499    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2" podUID="2741d67f-f693-4bb4-99fe-c169b380e663" containerName="payload-processing" containerID="cri-o://4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed" gracePeriod=25
Oct 31 04:09:01.678427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.678377    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k" podUID="f2dccd3f-32eb-4725-8957-7cd1d61fe22a" containerName="payload-pre-processing" containerID="cri-o://b7213fb3dad5f79c99cbd5ab3a8894459c4297383caf28a6e59a2eb1fa412828" gracePeriod=25
Oct 31 04:09:01.854519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.854443    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:09:01.938171 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.938141    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst\" (UniqueName: \"kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst\") pod \"2741d67f-f693-4bb4-99fe-c169b380e663\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") "
Oct 31 04:09:01.938403 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.938243    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume\") pod \"2741d67f-f693-4bb4-99fe-c169b380e663\" (UID: \"2741d67f-f693-4bb4-99fe-c169b380e663\") "
Oct 31 04:09:01.939093 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.939071    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume" pod "2741d67f-f693-4bb4-99fe-c169b380e663" (UID: "2741d67f-f693-4bb4-99fe-c169b380e663"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:01.944478 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.944439    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst" pod "2741d67f-f693-4bb4-99fe-c169b380e663" (UID: "2741d67f-f693-4bb4-99fe-c169b380e663"). InnerVolumeSpecName "kube-api-access-phpst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:01.944983 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.944948    5906 generic.go:356] "Generic (PLEG): container finished" podID="2741d67f-f693-4bb4-99fe-c169b380e663" containerID="4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed" exitCode=0
Oct 31 04:09:01.945226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.945108    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2" event={"ID":"2741d67f-f693-4bb4-99fe-c169b380e663","Type":"ContainerDied","Data":"4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed"}
Oct 31 04:09:01.945226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.945163    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2" event={"ID":"2741d67f-f693-4bb4-99fe-c169b380e663","Type":"ContainerDied","Data":"e141a9ef1b7f158351927a42346744dc4b5a6af7b271128f4ced1daeb837ac4a"}
Oct 31 04:09:01.945226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.945188    5906 scope.go:122] "RemoveContainer" containerID="4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed"
Oct 31 04:09:01.945418 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.945403    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"
Oct 31 04:09:01.950765 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.950732    5906 generic.go:356] "Generic (PLEG): container finished" podID="f2dccd3f-32eb-4725-8957-7cd1d61fe22a" containerID="b7213fb3dad5f79c99cbd5ab3a8894459c4297383caf28a6e59a2eb1fa412828" exitCode=0
Oct 31 04:09:01.950902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.950816    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k" event={"ID":"f2dccd3f-32eb-4725-8957-7cd1d61fe22a","Type":"ContainerDied","Data":"b7213fb3dad5f79c99cbd5ab3a8894459c4297383caf28a6e59a2eb1fa412828"}
Oct 31 04:09:01.965102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.965079    5906 scope.go:122] "RemoveContainer" containerID="4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed"
Oct 31 04:09:01.965689 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:01.965659    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed\": container with ID starting with 4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed not found: ID does not exist" containerID="4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed"
Oct 31 04:09:01.965807 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.965694    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed"} err="failed to get container status \"4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed\": rpc error: code = NotFound desc = could not find container \"4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed\": container with ID starting with 4fe3762eb657d04d30b49268604bddfc15bdc712d6dbcb776fc61eca5b8b8fed not found: ID does not exist"
Oct 31 04:09:01.969591 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:01.969566    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:09:02.028778 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.028734    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"]
Oct 31 04:09:02.028961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.028935    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" podUID="02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" containerName="maas-api" containerID="cri-o://8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c" gracePeriod=30
Oct 31 04:09:02.039534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.039497    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz\" (UniqueName: \"kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz\") pod \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") "
Oct 31 04:09:02.039695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.039618    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume\") pod \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\" (UID: \"f2dccd3f-32eb-4725-8957-7cd1d61fe22a\") "
Oct 31 04:09:02.040028 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.039998    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume" pod "f2dccd3f-32eb-4725-8957-7cd1d61fe22a" (UID: "f2dccd3f-32eb-4725-8957-7cd1d61fe22a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:02.040300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.040282    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phpst\" (UniqueName: \"kubernetes.io/projected/2741d67f-f693-4bb4-99fe-c169b380e663-kube-api-access-phpst\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.040388 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.040305    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.040388 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.040320    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2741d67f-f693-4bb4-99fe-c169b380e663-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.043243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.043170    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz" pod "f2dccd3f-32eb-4725-8957-7cd1d61fe22a" (UID: "f2dccd3f-32eb-4725-8957-7cd1d61fe22a"). InnerVolumeSpecName "kube-api-access-9hbhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:02.119669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.119584    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:02.142333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.142301    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hbhz\" (UniqueName: \"kubernetes.io/projected/f2dccd3f-32eb-4725-8957-7cd1d61fe22a-kube-api-access-9hbhz\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.214257 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.214223    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"]
Oct 31 04:09:02.220066 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.218532    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-5a5baa-6c5d4d45f-xh6t2"]
Oct 31 04:09:02.327114 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.327086    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:09:02.353089 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.353052    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2741d67f-f693-4bb4-99fe-c169b380e663" path="/var/lib/kubelet/pods/2741d67f-f693-4bb4-99fe-c169b380e663/volumes"
Oct 31 04:09:02.494899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.494864    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs\") pod \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") "
Oct 31 04:09:02.497270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.497238    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca\") pod \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") "
Oct 31 04:09:02.497409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.497328    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm\" (UniqueName: \"kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm\") pod \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") "
Oct 31 04:09:02.497472 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.497428    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\") pod \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\" (UID: \"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e\") "
Oct 31 04:09:02.497599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.497571    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca" pod "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" (UID: "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:02.498277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.498250    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.499738 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.499698    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs" pod "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" (UID: "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:09:02.500314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.500288    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls" pod "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" (UID: "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:09:02.500612 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.500591    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm" pod "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" (UID: "02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e"). InnerVolumeSpecName "kube-api-access-crjbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:02.602259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.601590    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crjbm\" (UniqueName: \"kubernetes.io/projected/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-kube-api-access-crjbm\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.602259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.601631    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.602259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.601646    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:02.965280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.964352    5906 generic.go:356] "Generic (PLEG): container finished" podID="02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" containerID="8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c" exitCode=0
Oct 31 04:09:02.965280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.964497    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" event={"ID":"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e","Type":"ContainerDied","Data":"8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c"}
Oct 31 04:09:02.965280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.964549    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6" event={"ID":"02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e","Type":"ContainerDied","Data":"f93ae3a69110a614a518b8b782961ade87fcbc69e3a19a3b6367e989666edabd"}
Oct 31 04:09:02.965280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.964582    5906 scope.go:122] "RemoveContainer" containerID="8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c"
Oct 31 04:09:02.965280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.964689    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"
Oct 31 04:09:02.986267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.975453    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k" event={"ID":"f2dccd3f-32eb-4725-8957-7cd1d61fe22a","Type":"ContainerDied","Data":"dd18c6ad3208071c8d0cf7be4826ccdd904a37556285a8d447ad92cd92a90905"}
Oct 31 04:09:02.986267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:02.975590    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"
Oct 31 04:09:03.001166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.001133    5906 scope.go:122] "RemoveContainer" containerID="8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c"
Oct 31 04:09:03.001779 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:03.001664    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c\": container with ID starting with 8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c not found: ID does not exist" containerID="8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c"
Oct 31 04:09:03.001779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.001706    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c"} err="failed to get container status \"8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c\": rpc error: code = NotFound desc = could not find container \"8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c\": container with ID starting with 8dd84ea4472dd13311ac81aa731c624dd574a85a8b86d0301a675ba08728502c not found: ID does not exist"
Oct 31 04:09:03.001779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.001728    5906 scope.go:122] "RemoveContainer" containerID="b7213fb3dad5f79c99cbd5ab3a8894459c4297383caf28a6e59a2eb1fa412828"
Oct 31 04:09:03.142693 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.142655    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"]
Oct 31 04:09:03.191220 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.191177    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5a5baa-7bb4fb7b7-wg42k"]
Oct 31 04:09:03.256540 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.256506    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"]
Oct 31 04:09:03.260086 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.260058    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-5a5baa-65589cfdf9-dvck6"]
Oct 31 04:09:03.546224 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.546121    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:03.806800 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.806689    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:03.945520 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:03.945488    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:04.354334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:04.354299    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" path="/var/lib/kubelet/pods/02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e/volumes"
Oct 31 04:09:04.354832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:04.354810    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2dccd3f-32eb-4725-8957-7cd1d61fe22a" path="/var/lib/kubelet/pods/f2dccd3f-32eb-4725-8957-7cd1d61fe22a/volumes"
Oct 31 04:09:04.475393 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:04.475333    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:06.381723 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:06.381680    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:09:08.495512 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:08.495467    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:10.502170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:10.502112    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:10.638683 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:10.638641    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:13.332371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:13.332308    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:17.798880 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:17.798847    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:20.828187 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:20.828155    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:20.834086 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:20.834054    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"]
Oct 31 04:09:20.834297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:20.834278    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" containerID="cri-o://c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76" gracePeriod=30
Oct 31 04:09:21.613990 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:21.613959    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:21.756337 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:21.756306    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:22.384559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:22.384524    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:25.445283 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:25.445246    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:09:25.445669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:25.445304    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:09:26.112503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.112444    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:09:26.139181 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.139150    5906 generic.go:356] "Generic (PLEG): container finished" podID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerID="c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76" exitCode=0
Oct 31 04:09:26.139384 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.139200    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"
Oct 31 04:09:26.139384 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.139251    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" event={"ID":"c44d273e-2eb6-41a7-a91e-35905e76ab88","Type":"ContainerDied","Data":"c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76"}
Oct 31 04:09:26.139384 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.139303    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z" event={"ID":"c44d273e-2eb6-41a7-a91e-35905e76ab88","Type":"ContainerDied","Data":"175e20c7c2e52d830bc1ca4f2c5a654b4fbcffcef24c520c6eaffaefb0a8e99a"}
Oct 31 04:09:26.139384 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.139326    5906 scope.go:122] "RemoveContainer" containerID="c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76"
Oct 31 04:09:26.149101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.149082    5906 scope.go:122] "RemoveContainer" containerID="08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"
Oct 31 04:09:26.158406 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.158371    5906 scope.go:122] "RemoveContainer" containerID="c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76"
Oct 31 04:09:26.158730 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:26.158705    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76\": container with ID starting with c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76 not found: ID does not exist" containerID="c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76"
Oct 31 04:09:26.158805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.158738    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76"} err="failed to get container status \"c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76\": rpc error: code = NotFound desc = could not find container \"c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76\": container with ID starting with c4d493c87168183d9221aaaf9ada09a2752d4f189447b9e8c1dcdee0d0a70e76 not found: ID does not exist"
Oct 31 04:09:26.158805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.158759    5906 scope.go:122] "RemoveContainer" containerID="08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"
Oct 31 04:09:26.159020 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:26.159003    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c\": container with ID starting with 08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c not found: ID does not exist" containerID="08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"
Oct 31 04:09:26.159076 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.159029    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c"} err="failed to get container status \"08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c\": rpc error: code = NotFound desc = could not find container \"08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c\": container with ID starting with 08e881e73e73ea8051c339551cf0f4ba126c512a3d6b80cad0c0f16f8d05936c not found: ID does not exist"
Oct 31 04:09:26.239718 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.239683    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.239918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.239749    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.239918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.239781    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.239918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.239835    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.239918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.239860    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.240144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.239938    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.240144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240038    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.240144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240125    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:26.240144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240137    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.240388 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240284    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token\") pod \"c44d273e-2eb6-41a7-a91e-35905e76ab88\" (UID: \"c44d273e-2eb6-41a7-a91e-35905e76ab88\") "
Oct 31 04:09:26.240439 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240395    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:26.240665 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240643    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:26.241000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240809    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:26.241000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240827    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:26.241000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240948    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.241000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240969    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.241000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.240988    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.241000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.241005    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c44d273e-2eb6-41a7-a91e-35905e76ab88-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.241451 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.241020    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.242101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.242071    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:26.242287 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.242264    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:09:26.242371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.242359    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "kube-api-access-s4zlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:26.242748 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.242730    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token" pod "c44d273e-2eb6-41a7-a91e-35905e76ab88" (UID: "c44d273e-2eb6-41a7-a91e-35905e76ab88"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:26.342675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.342641    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.342675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.342671    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.342675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.342680    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c44d273e-2eb6-41a7-a91e-35905e76ab88-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.342928 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.342689    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4zlt\" (UniqueName: \"kubernetes.io/projected/c44d273e-2eb6-41a7-a91e-35905e76ab88-kube-api-access-s4zlt\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:26.566862 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.566713    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"]
Oct 31 04:09:26.567772 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:26.567751    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-5a5baa-openshift-default-b59f5db5-9vv2z"]
Oct 31 04:09:27.268230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:27.265438    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:27.743789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:27.743749    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:28.352796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:28.352765    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" path="/var/lib/kubelet/pods/c44d273e-2eb6-41a7-a91e-35905e76ab88/volumes"
Oct 31 04:09:31.072347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:31.072306    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:32.498083 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:32.498038    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:32.706802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:32.706767    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:32.850008 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:32.849642    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:33.440398 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:33.440368    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:33.598260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:33.598222    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:36.382016 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:36.381979    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:09:37.395661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:37.395619    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:41.989076 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:41.989034    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:43.406560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:43.406511    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:44.936885 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:44.936850    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"]
Oct 31 04:09:44.937341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:44.937073    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" containerID="cri-o://03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3" gracePeriod=30
Oct 31 04:09:45.925702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:45.925667    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:47.383000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:47.382957    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:47.600362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:47.600323    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:47.739293 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:47.739096    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:48.306321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:48.306289    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:49.554068 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:49.554030    5906 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:09:49.554505 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:49.554099    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:09:49.835661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:49.835560    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:49.979840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:49.979807    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:50.180402 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.180356    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:50.196868 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.196840    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"]
Oct 31 04:09:50.197079 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.197055    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" containerID="cri-o://d38bd453e1f723d2c1b1b540c6f8539d198b1a8065ef990a8fe7b8722e45dfaf" gracePeriod=30
Oct 31 04:09:50.206444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.206418    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 04:09:50.320696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.320650    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.321597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.321568    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.322267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.322244    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.322732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.322712    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.323347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.323327    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.323802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.323784    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.334667 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.322041    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:50.334667 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.322643    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:50.334667 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.323174    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:50.334667 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.334317    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:50.334667 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.334500    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "kube-api-access-gmtdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:50.335503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.335476    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.335765 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.335747    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.345652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.345610    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo\") pod \"a7064811-4b4b-4608-aa2a-5469c6d2571f\" (UID: \"a7064811-4b4b-4608-aa2a-5469c6d2571f\") "
Oct 31 04:09:50.346535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.346502    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmtdv\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-kube-api-access-gmtdv\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.346535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.346533    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.346796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.346548    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.346796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.346562    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.346796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.346574    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7064811-4b4b-4608-aa2a-5469c6d2571f-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.346981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.338188    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:50.347056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.338679    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:50.347056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.345631    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:50.349551 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.349523    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo" pod "a7064811-4b4b-4608-aa2a-5469c6d2571f" (UID: "a7064811-4b4b-4608-aa2a-5469c6d2571f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:09:50.353309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.353276    5906 generic.go:356] "Generic (PLEG): container finished" podID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerID="03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3" exitCode=0
Oct 31 04:09:50.353443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.353406    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"
Oct 31 04:09:50.359999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.359734    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" event={"ID":"a7064811-4b4b-4608-aa2a-5469c6d2571f","Type":"ContainerDied","Data":"03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3"}
Oct 31 04:09:50.359999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.359808    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct" event={"ID":"a7064811-4b4b-4608-aa2a-5469c6d2571f","Type":"ContainerDied","Data":"3ff71cfe5600b8dd172e4b29a3686cd70bedbd85238807c07d2535b4adbb1bbd"}
Oct 31 04:09:50.359999 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.359842    5906 scope.go:122] "RemoveContainer" containerID="03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3"
Oct 31 04:09:50.373200 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.373181    5906 scope.go:122] "RemoveContainer" containerID="03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3"
Oct 31 04:09:50.373546 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:09:50.373526    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3\": container with ID starting with 03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3 not found: ID does not exist" containerID="03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3"
Oct 31 04:09:50.373596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.373556    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3"} err="failed to get container status \"03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3\": rpc error: code = NotFound desc = could not find container \"03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3\": container with ID starting with 03bf72b4d831c9552a796bc252c78e8499a5e7ce3d2ad733dcffeb92af6309d3 not found: ID does not exist"
Oct 31 04:09:50.447778 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.447737    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.447778 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.447772    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.447778 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.447782    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.448030 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.447790    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7064811-4b4b-4608-aa2a-5469c6d2571f-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:50.589552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.589519    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"]
Oct 31 04:09:50.593236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:50.593181    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-4rkct"]
Oct 31 04:09:52.352371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:52.352336    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" path="/var/lib/kubelet/pods/a7064811-4b4b-4608-aa2a-5469c6d2571f/volumes"
Oct 31 04:09:52.405889 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:52.405857    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:53.048266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:53.048228    5906 patch_prober.go:29] interesting pod/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:09:53.048453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:53.048289    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:09:54.540577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:54.540543    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:54.680790 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:54.680741    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:54.890403 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:54.890304    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"]
Oct 31 04:09:54.890594 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:54.890569    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy" containerID="cri-o://693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec" gracePeriod=30
Oct 31 04:09:54.895433 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:54.895397    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:55.353318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.353272    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:55.394271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.394234    5906 generic.go:356] "Generic (PLEG): container finished" podID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerID="d38bd453e1f723d2c1b1b540c6f8539d198b1a8065ef990a8fe7b8722e45dfaf" exitCode=0
Oct 31 04:09:55.394652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.394259    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" event={"ID":"5e688e64-20e8-471d-97b5-55ebadcaa984","Type":"ContainerDied","Data":"d38bd453e1f723d2c1b1b540c6f8539d198b1a8065ef990a8fe7b8722e45dfaf"}
Oct 31 04:09:55.394814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.394734    5906 scope.go:122] "RemoveContainer" containerID="ea07630487e90ba96ba551eb33d94857af930af75b8ae58fb4ae59fd213b30b0"
Oct 31 04:09:55.485435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.485406    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:09:55.545267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545231    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545370    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545412    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545439    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545476    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545513    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545577    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545610    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545670    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:55.545708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545677    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq\") pod \"5e688e64-20e8-471d-97b5-55ebadcaa984\" (UID: \"5e688e64-20e8-471d-97b5-55ebadcaa984\") "
Oct 31 04:09:55.546243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545828    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:55.546243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545841    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:55.546243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545856    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:55.546243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.545975    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:55.546409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.546259    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e688e64-20e8-471d-97b5-55ebadcaa984-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.546409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.546279    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.546409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.546294    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.546409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.546307    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.546409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.546320    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.547996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.547974    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:55.548088 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.548027    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "kube-api-access-z6qbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:55.548184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.548161    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:09:55.548251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.548167    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy" pod "5e688e64-20e8-471d-97b5-55ebadcaa984" (UID: "5e688e64-20e8-471d-97b5-55ebadcaa984"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:09:55.647481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.647433    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.647481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.647467    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.647481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.647481    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e688e64-20e8-471d-97b5-55ebadcaa984-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:55.647818 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:55.647496    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6qbq\" (UniqueName: \"kubernetes.io/projected/5e688e64-20e8-471d-97b5-55ebadcaa984-kube-api-access-z6qbq\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:56.402039 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:56.402011    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"
Oct 31 04:09:56.402226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:56.401998    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq" event={"ID":"5e688e64-20e8-471d-97b5-55ebadcaa984","Type":"ContainerDied","Data":"b84ce3bea6b41c834a9a3652ade6c7fe93e93f1dbd4267c020bba1d7742dd75e"}
Oct 31 04:09:56.402226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:56.402154    5906 scope.go:122] "RemoveContainer" containerID="d38bd453e1f723d2c1b1b540c6f8539d198b1a8065ef990a8fe7b8722e45dfaf"
Oct 31 04:09:56.625380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:56.625301    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"]
Oct 31 04:09:56.630995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:56.630964    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-3473a4-openshift-default-d98b69b47-dzthq"]
Oct 31 04:09:57.141315 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:57.141284    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:09:58.353563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:09:58.353528    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" path="/var/lib/kubelet/pods/5e688e64-20e8-471d-97b5-55ebadcaa984/volumes"
Oct 31 04:10:00.186620 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.186592    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:10:00.313821 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.313787    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.313844    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.313922    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314163 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314043    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314278    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:00.314469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314332    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:00.314469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314079    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314469 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314401    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:00.314813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314596    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:00.314813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314669    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314711    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.314813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314795    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.315016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.314853    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token\") pod \"856a336a-5439-48b5-af95-01572728f411\" (UID: \"856a336a-5439-48b5-af95-01572728f411\") "
Oct 31 04:10:00.315874 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.315785    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/856a336a-5439-48b5-af95-01572728f411-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.315874 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.315844    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.315874 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.315862    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.315874 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.315876    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.316147 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.315789    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:00.317921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.317894    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:00.318319 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.318292    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:00.318319 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.318295    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "kube-api-access-jwzgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:00.318492 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.318333    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo" pod "856a336a-5439-48b5-af95-01572728f411" (UID: "856a336a-5439-48b5-af95-01572728f411"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:10:00.318492 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.318446    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"]
Oct 31 04:10:00.417045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.416966    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwzgx\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-kube-api-access-jwzgx\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.417243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.417070    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.417243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.417101    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/856a336a-5439-48b5-af95-01572728f411-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.417243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.417115    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/856a336a-5439-48b5-af95-01572728f411-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.417243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.417129    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/856a336a-5439-48b5-af95-01572728f411-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:00.428827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.428790    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:00.429851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.429821    5906 generic.go:356] "Generic (PLEG): container finished" podID="856a336a-5439-48b5-af95-01572728f411" containerID="693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec" exitCode=0
Oct 31 04:10:00.429963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.429872    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"
Oct 31 04:10:00.429963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.429901    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" event={"ID":"856a336a-5439-48b5-af95-01572728f411","Type":"ContainerDied","Data":"693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec"}
Oct 31 04:10:00.429963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.429957    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx" event={"ID":"856a336a-5439-48b5-af95-01572728f411","Type":"ContainerDied","Data":"4af2520bac72a89a849d2dae5fbb9d51ce1e496d3b176fae341345c48175958e"}
Oct 31 04:10:00.430079 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.429985    5906 scope.go:122] "RemoveContainer" containerID="693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec"
Oct 31 04:10:00.439715 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.439697    5906 scope.go:122] "RemoveContainer" containerID="afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"
Oct 31 04:10:00.454134 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.454108    5906 scope.go:122] "RemoveContainer" containerID="693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec"
Oct 31 04:10:00.454809 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:00.454787    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec\": container with ID starting with 693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec not found: ID does not exist" containerID="693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec"
Oct 31 04:10:00.454877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.454820    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec"} err="failed to get container status \"693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec\": rpc error: code = NotFound desc = could not find container \"693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec\": container with ID starting with 693980ddd5331a137e7dcf64eecc7532ff9a9a3fe9b9284bec35bc2519ab68ec not found: ID does not exist"
Oct 31 04:10:00.454877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.454843    5906 scope.go:122] "RemoveContainer" containerID="afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"
Oct 31 04:10:00.455110 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:00.455094    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845\": container with ID starting with afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845 not found: ID does not exist" containerID="afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"
Oct 31 04:10:00.455153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.455115    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845"} err="failed to get container status \"afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845\": rpc error: code = NotFound desc = could not find container \"afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845\": container with ID starting with afee7870575be3af367ea41e0f7d32657970ac86ce7f06f883e721522a45d845 not found: ID does not exist"
Oct 31 04:10:00.600942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.600903    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"]
Oct 31 04:10:00.605038 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:00.605008    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-288b63-openshift-default-76d49649f9-zkmqx"]
Oct 31 04:10:02.033673 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:02.033635    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:02.170837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:02.170799    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"]
Oct 31 04:10:02.281994 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:02.281946    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"]
Oct 31 04:10:02.352183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:02.352101    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="856a336a-5439-48b5-af95-01572728f411" path="/var/lib/kubelet/pods/856a336a-5439-48b5-af95-01572728f411/volumes"
Oct 31 04:10:02.582446 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:02.582399    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:02.735640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:02.735543    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:06.381891 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:06.381856    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:10:06.606240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:06.606189    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:07.171682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.171635    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc" podUID="57f0f9df-fecb-415f-8cac-441031401154" containerName="payload-processing" containerID="cri-o://f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87" gracePeriod=25
Oct 31 04:10:07.282787 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.282726    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx" podUID="9a787113-91b9-4814-8828-501ac0bb5f76" containerName="payload-pre-processing" containerID="cri-o://6fbe651456b978fe270b11dc1c6156a0124228f55ddb4745ef5d7c2eb7412179" gracePeriod=25
Oct 31 04:10:07.444250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.443641    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"]
Oct 31 04:10:07.456437 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.456407    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 04:10:07.475440 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.475400    5906 generic.go:356] "Generic (PLEG): container finished" podID="57f0f9df-fecb-415f-8cac-441031401154" containerID="f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87" exitCode=0
Oct 31 04:10:07.475614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.475467    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"
Oct 31 04:10:07.475614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.475479    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc" event={"ID":"57f0f9df-fecb-415f-8cac-441031401154","Type":"ContainerDied","Data":"f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87"}
Oct 31 04:10:07.475614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.475553    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc" event={"ID":"57f0f9df-fecb-415f-8cac-441031401154","Type":"ContainerDied","Data":"1e35c1c6028d800de7d54150029f54e2600c4c61d9d64e908eb11c29589312e5"}
Oct 31 04:10:07.475614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.475587    5906 scope.go:122] "RemoveContainer" containerID="f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87"
Oct 31 04:10:07.477735 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.477646    5906 generic.go:356] "Generic (PLEG): container finished" podID="9a787113-91b9-4814-8828-501ac0bb5f76" containerID="6fbe651456b978fe270b11dc1c6156a0124228f55ddb4745ef5d7c2eb7412179" exitCode=0
Oct 31 04:10:07.477735 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.477681    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx" event={"ID":"9a787113-91b9-4814-8828-501ac0bb5f76","Type":"ContainerDied","Data":"6fbe651456b978fe270b11dc1c6156a0124228f55ddb4745ef5d7c2eb7412179"}
Oct 31 04:10:07.494727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.494695    5906 scope.go:122] "RemoveContainer" containerID="f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87"
Oct 31 04:10:07.495281 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:07.495255    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87\": container with ID starting with f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87 not found: ID does not exist" containerID="f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87"
Oct 31 04:10:07.495406 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.495297    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87"} err="failed to get container status \"f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87\": rpc error: code = NotFound desc = could not find container \"f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87\": container with ID starting with f5b770f7519afe770c8a5451e3babdaca8c002ecd60492ef29db68357e271d87 not found: ID does not exist"
Oct 31 04:10:07.574859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.574834    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 04:10:07.626696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.626645    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume\") pod \"57f0f9df-fecb-415f-8cac-441031401154\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") "
Oct 31 04:10:07.626891 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.626768    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg\" (UniqueName: \"kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg\") pod \"57f0f9df-fecb-415f-8cac-441031401154\" (UID: \"57f0f9df-fecb-415f-8cac-441031401154\") "
Oct 31 04:10:07.627126 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.627094    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume" pod "57f0f9df-fecb-415f-8cac-441031401154" (UID: "57f0f9df-fecb-415f-8cac-441031401154"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:07.627506 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.627483    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57f0f9df-fecb-415f-8cac-441031401154-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:07.629870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.629842    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg" pod "57f0f9df-fecb-415f-8cac-441031401154" (UID: "57f0f9df-fecb-415f-8cac-441031401154"). InnerVolumeSpecName "kube-api-access-s5zgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:07.684303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.684182    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:07.728717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.728675    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm\" (UniqueName: \"kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm\") pod \"9a787113-91b9-4814-8828-501ac0bb5f76\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") "
Oct 31 04:10:07.728922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.728767    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume\") pod \"9a787113-91b9-4814-8828-501ac0bb5f76\" (UID: \"9a787113-91b9-4814-8828-501ac0bb5f76\") "
Oct 31 04:10:07.729284 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.729249    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume" pod "9a787113-91b9-4814-8828-501ac0bb5f76" (UID: "9a787113-91b9-4814-8828-501ac0bb5f76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:07.729459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.729432    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5zgg\" (UniqueName: \"kubernetes.io/projected/57f0f9df-fecb-415f-8cac-441031401154-kube-api-access-s5zgg\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:07.729556 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.729465    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a787113-91b9-4814-8828-501ac0bb5f76-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:07.732958 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.732924    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm" pod "9a787113-91b9-4814-8828-501ac0bb5f76" (UID: "9a787113-91b9-4814-8828-501ac0bb5f76"). InnerVolumeSpecName "kube-api-access-5l9zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:07.825843 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.825808    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:07.831162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.831135    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l9zm\" (UniqueName: \"kubernetes.io/projected/9a787113-91b9-4814-8828-501ac0bb5f76-kube-api-access-5l9zm\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:07.928012 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.927969    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"]
Oct 31 04:10:07.931621 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:07.931590    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w5-938507-7855c7c4f5-qlqbc"]
Oct 31 04:10:08.352418 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:08.352384    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57f0f9df-fecb-415f-8cac-441031401154" path="/var/lib/kubelet/pods/57f0f9df-fecb-415f-8cac-441031401154/volumes"
Oct 31 04:10:08.484466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:08.484389    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"
Oct 31 04:10:08.485513 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:08.484384    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx" event={"ID":"9a787113-91b9-4814-8828-501ac0bb5f76","Type":"ContainerDied","Data":"51d7d6b5e65aeb5e6e5764ae2b317076b77712d5a48430c1e41d9a3b7bc69182"}
Oct 31 04:10:08.485513 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:08.484539    5906 scope.go:122] "RemoveContainer" containerID="6fbe651456b978fe270b11dc1c6156a0124228f55ddb4745ef5d7c2eb7412179"
Oct 31 04:10:08.654280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:08.654244    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"]
Oct 31 04:10:08.659750 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:08.659719    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w5-938507-bb889d647-qfvhx"]
Oct 31 04:10:10.352413 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:10.352380    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a787113-91b9-4814-8828-501ac0bb5f76" path="/var/lib/kubelet/pods/9a787113-91b9-4814-8828-501ac0bb5f76/volumes"
Oct 31 04:10:11.158162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:11.158121    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:12.444770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:12.444725    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" podUID="dd808397-af2b-4790-9325-b936b3a052de" containerName="payload-pre-processing" containerID="cri-o://518296e96a1ae94ff7b12587db668a8e0d86bfb93f198f8e05a6df0e37ba0b5b" gracePeriod=25
Oct 31 04:10:13.222632 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:13.222594    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" podUID="dd808397-af2b-4790-9325-b936b3a052de" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:10:14.977996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:14.977959    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"]
Oct 31 04:10:14.978656 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:14.978389    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl" podUID="bc17dd46-e662-49bc-8994-a73af37493a3" containerName="maas-api" containerID="cri-o://4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4" gracePeriod=30
Oct 31 04:10:15.052395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.052346    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:15.286405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.286377    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:10:15.344004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.343960    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main" containerID="cri-o://be9f68d3664ec98ab639a4bf1fb577a5cc8ca157587d5ec98fc2a50c09539111" gracePeriod=45
Oct 31 04:10:15.434829 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.434793    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs\") pod \"bc17dd46-e662-49bc-8994-a73af37493a3\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") "
Oct 31 04:10:15.435025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.434848    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls\") pod \"bc17dd46-e662-49bc-8994-a73af37493a3\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") "
Oct 31 04:10:15.435025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.434885    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca\") pod \"bc17dd46-e662-49bc-8994-a73af37493a3\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") "
Oct 31 04:10:15.435025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.434974    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x\" (UniqueName: \"kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x\") pod \"bc17dd46-e662-49bc-8994-a73af37493a3\" (UID: \"bc17dd46-e662-49bc-8994-a73af37493a3\") "
Oct 31 04:10:15.435574 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.435548    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca" pod "bc17dd46-e662-49bc-8994-a73af37493a3" (UID: "bc17dd46-e662-49bc-8994-a73af37493a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:15.437721 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.437677    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls" pod "bc17dd46-e662-49bc-8994-a73af37493a3" (UID: "bc17dd46-e662-49bc-8994-a73af37493a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:15.437835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.437739    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x" pod "bc17dd46-e662-49bc-8994-a73af37493a3" (UID: "bc17dd46-e662-49bc-8994-a73af37493a3"). InnerVolumeSpecName "kube-api-access-4279x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:15.437896 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.437839    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs" pod "bc17dd46-e662-49bc-8994-a73af37493a3" (UID: "bc17dd46-e662-49bc-8994-a73af37493a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:15.535257 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.534572    5906 generic.go:356] "Generic (PLEG): container finished" podID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerID="be9f68d3664ec98ab639a4bf1fb577a5cc8ca157587d5ec98fc2a50c09539111" exitCode=0
Oct 31 04:10:15.535257 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.534810    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" event={"ID":"7f91b7ef-f035-4f81-951d-9dc17c538592","Type":"ContainerDied","Data":"be9f68d3664ec98ab639a4bf1fb577a5cc8ca157587d5ec98fc2a50c09539111"}
Oct 31 04:10:15.536407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.536368    5906 generic.go:356] "Generic (PLEG): container finished" podID="bc17dd46-e662-49bc-8994-a73af37493a3" containerID="4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4" exitCode=0
Oct 31 04:10:15.536558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.536420    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl" event={"ID":"bc17dd46-e662-49bc-8994-a73af37493a3","Type":"ContainerDied","Data":"4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4"}
Oct 31 04:10:15.536558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.536462    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl" event={"ID":"bc17dd46-e662-49bc-8994-a73af37493a3","Type":"ContainerDied","Data":"a1e42c731e7269e01183dab7b669cd16fab76a196c66dc0bb0bc234358f12164"}
Oct 31 04:10:15.536558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.536488    5906 scope.go:122] "RemoveContainer" containerID="4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4"
Oct 31 04:10:15.536742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.536622    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"
Oct 31 04:10:15.537633 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.537605    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.537754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.537638    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bc17dd46-e662-49bc-8994-a73af37493a3-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.537754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.537655    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bc17dd46-e662-49bc-8994-a73af37493a3-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.537754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.537671    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4279x\" (UniqueName: \"kubernetes.io/projected/bc17dd46-e662-49bc-8994-a73af37493a3-kube-api-access-4279x\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.552616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.552591    5906 scope.go:122] "RemoveContainer" containerID="4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4"
Oct 31 04:10:15.552995 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:15.552963    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4\": container with ID starting with 4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4 not found: ID does not exist" containerID="4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4"
Oct 31 04:10:15.553087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.553005    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4"} err="failed to get container status \"4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4\": rpc error: code = NotFound desc = could not find container \"4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4\": container with ID starting with 4e2af9a1541939b45cfb156b448584fdaec47086a9cd6b23037499466c7bbef4 not found: ID does not exist"
Oct 31 04:10:15.597054 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.597028    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 04:10:15.706658 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.706625    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"]
Oct 31 04:10:15.712308 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.712281    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-lflsl"]
Oct 31 04:10:15.739161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739132    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir\") pod \"7f91b7ef-f035-4f81-951d-9dc17c538592\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") "
Oct 31 04:10:15.739367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739176    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm\") pod \"7f91b7ef-f035-4f81-951d-9dc17c538592\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") "
Oct 31 04:10:15.739367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739222    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs\" (UniqueName: \"kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs\") pod \"7f91b7ef-f035-4f81-951d-9dc17c538592\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") "
Oct 31 04:10:15.739367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739286    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache\") pod \"7f91b7ef-f035-4f81-951d-9dc17c538592\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") "
Oct 31 04:10:15.739367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739308    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s\" (UniqueName: \"kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s\") pod \"7f91b7ef-f035-4f81-951d-9dc17c538592\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") "
Oct 31 04:10:15.739367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739364    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home\") pod \"7f91b7ef-f035-4f81-951d-9dc17c538592\" (UID: \"7f91b7ef-f035-4f81-951d-9dc17c538592\") "
Oct 31 04:10:15.739655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739546    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir" pod "7f91b7ef-f035-4f81-951d-9dc17c538592" (UID: "7f91b7ef-f035-4f81-951d-9dc17c538592"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.739655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739589    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache" pod "7f91b7ef-f035-4f81-951d-9dc17c538592" (UID: "7f91b7ef-f035-4f81-951d-9dc17c538592"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.739773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.739749    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home" pod "7f91b7ef-f035-4f81-951d-9dc17c538592" (UID: "7f91b7ef-f035-4f81-951d-9dc17c538592"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.740054 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.740034    5906 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-model-cache\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.740104 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.740062    5906 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-home\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.740104 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.740079    5906 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-tmp-dir\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.741574 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.741553    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm" pod "7f91b7ef-f035-4f81-951d-9dc17c538592" (UID: "7f91b7ef-f035-4f81-951d-9dc17c538592"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.741653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.741585    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs" pod "7f91b7ef-f035-4f81-951d-9dc17c538592" (UID: "7f91b7ef-f035-4f81-951d-9dc17c538592"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:15.741653 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.741614    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s" pod "7f91b7ef-f035-4f81-951d-9dc17c538592" (UID: "7f91b7ef-f035-4f81-951d-9dc17c538592"). InnerVolumeSpecName "kube-api-access-zd98s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:15.843973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.843873    5906 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f91b7ef-f035-4f81-951d-9dc17c538592-dshm\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.843973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.843915    5906 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f91b7ef-f035-4f81-951d-9dc17c538592-tls-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.843973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:15.843930    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd98s\" (UniqueName: \"kubernetes.io/projected/7f91b7ef-f035-4f81-951d-9dc17c538592-kube-api-access-zd98s\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:16.352727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.352691    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc17dd46-e662-49bc-8994-a73af37493a3" path="/var/lib/kubelet/pods/bc17dd46-e662-49bc-8994-a73af37493a3/volumes"
Oct 31 04:10:16.491101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.491040    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:16.544523 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.544493    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"
Oct 31 04:10:16.544523 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.544516    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f" event={"ID":"7f91b7ef-f035-4f81-951d-9dc17c538592","Type":"ContainerDied","Data":"5d9abc3c9d9a1238c56fad8b075461be5407410f511b98d18346b67c77ec2ebe"}
Oct 31 04:10:16.544764 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.544567    5906 scope.go:122] "RemoveContainer" containerID="be9f68d3664ec98ab639a4bf1fb577a5cc8ca157587d5ec98fc2a50c09539111"
Oct 31 04:10:16.713860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.713765    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:16.715390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.715359    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"]
Oct 31 04:10:16.719280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.719257    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-2-w0-ae8e27-kserve-6f8985757b-ggm2f"]
Oct 31 04:10:16.856986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:16.856921    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:18.352061 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:18.352028    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" path="/var/lib/kubelet/pods/7f91b7ef-f035-4f81-951d-9dc17c538592/volumes"
Oct 31 04:10:20.917077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:20.917049    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:21.386336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:21.386300    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:23.222500 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:23.222462    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" podUID="dd808397-af2b-4790-9325-b936b3a052de" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:10:24.434650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:24.434613    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:24.596587 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:24.596551    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:25.353851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.353812    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"]
Oct 31 04:10:25.354059 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.354016    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" podUID="d99c7b2d-bfad-45c9-9691-6e888fab2082" containerName="maas-api" containerID="cri-o://bc4b53ed8c923955cf0323ebefef85ce7d4ee8927e2017c00d9caffd45c26e3c" gracePeriod=30
Oct 31 04:10:25.440961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.440916    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:25.603552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.603332    5906 generic.go:356] "Generic (PLEG): container finished" podID="d99c7b2d-bfad-45c9-9691-6e888fab2082" containerID="bc4b53ed8c923955cf0323ebefef85ce7d4ee8927e2017c00d9caffd45c26e3c" exitCode=0
Oct 31 04:10:25.603552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.603411    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" event={"ID":"d99c7b2d-bfad-45c9-9691-6e888fab2082","Type":"ContainerDied","Data":"bc4b53ed8c923955cf0323ebefef85ce7d4ee8927e2017c00d9caffd45c26e3c"}
Oct 31 04:10:25.654910 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.654875    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 04:10:25.785436 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.785399    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\") pod \"d99c7b2d-bfad-45c9-9691-6e888fab2082\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") "
Oct 31 04:10:25.785619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.785512    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h\" (UniqueName: \"kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h\") pod \"d99c7b2d-bfad-45c9-9691-6e888fab2082\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") "
Oct 31 04:10:25.785619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.785551    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca\") pod \"d99c7b2d-bfad-45c9-9691-6e888fab2082\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") "
Oct 31 04:10:25.785619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.785592    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs\") pod \"d99c7b2d-bfad-45c9-9691-6e888fab2082\" (UID: \"d99c7b2d-bfad-45c9-9691-6e888fab2082\") "
Oct 31 04:10:25.785987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.785953    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca" pod "d99c7b2d-bfad-45c9-9691-6e888fab2082" (UID: "d99c7b2d-bfad-45c9-9691-6e888fab2082"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:25.788230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.788185    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls" pod "d99c7b2d-bfad-45c9-9691-6e888fab2082" (UID: "d99c7b2d-bfad-45c9-9691-6e888fab2082"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:25.788745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.788715    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs" pod "d99c7b2d-bfad-45c9-9691-6e888fab2082" (UID: "d99c7b2d-bfad-45c9-9691-6e888fab2082"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:25.788986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.788957    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h" pod "d99c7b2d-bfad-45c9-9691-6e888fab2082" (UID: "d99c7b2d-bfad-45c9-9691-6e888fab2082"). InnerVolumeSpecName "kube-api-access-sk55h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:25.887717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.887625    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:25.887717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.887664    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk55h\" (UniqueName: \"kubernetes.io/projected/d99c7b2d-bfad-45c9-9691-6e888fab2082-kube-api-access-sk55h\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:25.887717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.887674    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d99c7b2d-bfad-45c9-9691-6e888fab2082-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:25.887717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:25.887683    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d99c7b2d-bfad-45c9-9691-6e888fab2082-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:26.610133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.610046    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn" event={"ID":"d99c7b2d-bfad-45c9-9691-6e888fab2082","Type":"ContainerDied","Data":"6afdd07f73bd23cca93ac9d7e77f796cfbd86fef6dcac79f440f630916204ac2"}
Oct 31 04:10:26.610133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.610078    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"
Oct 31 04:10:26.610133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.610113    5906 scope.go:122] "RemoveContainer" containerID="bc4b53ed8c923955cf0323ebefef85ce7d4ee8927e2017c00d9caffd45c26e3c"
Oct 31 04:10:26.614614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.614589    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:26.761196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.760696    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:26.794464 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.794426    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"]
Oct 31 04:10:26.796466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:26.796440    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-938507-95cc7b96c-x4psn"]
Oct 31 04:10:27.256809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:27.256779    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:27.514721 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:27.514616    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:27.669227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:27.668926    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:28.086895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:28.086855    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:28.230329 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:28.230296    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:28.352401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:28.352304    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d99c7b2d-bfad-45c9-9691-6e888fab2082" path="/var/lib/kubelet/pods/d99c7b2d-bfad-45c9-9691-6e888fab2082/volumes"
Oct 31 04:10:32.197523 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.197492    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:32.672760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.672729    5906 generic.go:356] "Generic (PLEG): container finished" podID="dd808397-af2b-4790-9325-b936b3a052de" containerID="518296e96a1ae94ff7b12587db668a8e0d86bfb93f198f8e05a6df0e37ba0b5b" exitCode=0
Oct 31 04:10:32.672894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.672774    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" event={"ID":"dd808397-af2b-4790-9325-b936b3a052de","Type":"ContainerDied","Data":"518296e96a1ae94ff7b12587db668a8e0d86bfb93f198f8e05a6df0e37ba0b5b"}
Oct 31 04:10:32.702183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.702115    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 04:10:32.803771 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.803735    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr\" (UniqueName: \"kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr\") pod \"dd808397-af2b-4790-9325-b936b3a052de\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") "
Oct 31 04:10:32.803981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.803798    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume\") pod \"dd808397-af2b-4790-9325-b936b3a052de\" (UID: \"dd808397-af2b-4790-9325-b936b3a052de\") "
Oct 31 04:10:32.804227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.804186    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume" pod "dd808397-af2b-4790-9325-b936b3a052de" (UID: "dd808397-af2b-4790-9325-b936b3a052de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:32.804387 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.804370    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dd808397-af2b-4790-9325-b936b3a052de-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:32.805975 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.805951    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr" pod "dd808397-af2b-4790-9325-b936b3a052de" (UID: "dd808397-af2b-4790-9325-b936b3a052de"). InnerVolumeSpecName "kube-api-access-svqbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:32.905371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:32.905323    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svqbr\" (UniqueName: \"kubernetes.io/projected/dd808397-af2b-4790-9325-b936b3a052de-kube-api-access-svqbr\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:33.679175 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:33.679144    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"
Oct 31 04:10:33.679175 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:33.679153    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk" event={"ID":"dd808397-af2b-4790-9325-b936b3a052de","Type":"ContainerDied","Data":"38e5b87c48d897db4806a206293caf43a4a8961acf89c3378b9b1c78142a040b"}
Oct 31 04:10:33.679719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:33.679230    5906 scope.go:122] "RemoveContainer" containerID="518296e96a1ae94ff7b12587db668a8e0d86bfb93f198f8e05a6df0e37ba0b5b"
Oct 31 04:10:33.849057 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:33.849023    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"]
Oct 31 04:10:33.901171 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:33.901128    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-ae8e27-565f84b559-shbxk"]
Oct 31 04:10:34.352347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:34.352315    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd808397-af2b-4790-9325-b936b3a052de" path="/var/lib/kubelet/pods/dd808397-af2b-4790-9325-b936b3a052de/volumes"
Oct 31 04:10:34.409922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:34.409888    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:34.558548 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:34.558516    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:34.750407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:34.750372    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:34.762240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:34.762186    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"]
Oct 31 04:10:34.762473 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:34.762450    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" containerID="cri-o://6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996" gracePeriod=30
Oct 31 04:10:36.383279 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:36.383234    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:10:36.762905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.762867    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"]
Oct 31 04:10:36.763411 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763395    5906 cpu_manager.go:414] "containerMap: removing container" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy"
Oct 31 04:10:36.763461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763412    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy"
Oct 31 04:10:36.763461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763427    5906 cpu_manager.go:414] "containerMap: removing container" podUID="9e2ce747-c974-460c-b44d-c4e6f0c62734" containerName="maas-api"
Oct 31 04:10:36.763461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763432    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e2ce747-c974-460c-b44d-c4e6f0c62734" containerName="maas-api"
Oct 31 04:10:36.763461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763448    5906 cpu_manager.go:414] "containerMap: removing container" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy"
Oct 31 04:10:36.763461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763453    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy"
Oct 31 04:10:36.763461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763462    5906 cpu_manager.go:414] "containerMap: removing container" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763467    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763477    5906 cpu_manager.go:414] "containerMap: removing container" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763482    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763488    5906 cpu_manager.go:414] "containerMap: removing container" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763493    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763501    5906 cpu_manager.go:414] "containerMap: removing container" podUID="521f908c-8733-4b71-af51-4f0fcd339c09" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763505    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="521f908c-8733-4b71-af51-4f0fcd339c09" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763514    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerName="payload-pre-processing"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763519    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerName="payload-pre-processing"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763526    5906 cpu_manager.go:414] "containerMap: removing container" podUID="02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763530    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763540    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a79b0003-c031-4b13-9375-97a1d0ccefbb" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763545    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a79b0003-c031-4b13-9375-97a1d0ccefbb" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763552    5906 cpu_manager.go:414] "containerMap: removing container" podUID="bc17dd46-e662-49bc-8994-a73af37493a3" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763557    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc17dd46-e662-49bc-8994-a73af37493a3" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763563    5906 cpu_manager.go:414] "containerMap: removing container" podUID="57f0f9df-fecb-415f-8cac-441031401154" containerName="payload-processing"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763568    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57f0f9df-fecb-415f-8cac-441031401154" containerName="payload-processing"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763574    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f2dccd3f-32eb-4725-8957-7cd1d61fe22a" containerName="payload-pre-processing"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763579    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2dccd3f-32eb-4725-8957-7cd1d61fe22a" containerName="payload-pre-processing"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763585    5906 cpu_manager.go:414] "containerMap: removing container" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763589    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763596    5906 cpu_manager.go:414] "containerMap: removing container" podUID="7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763601    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763607    5906 cpu_manager.go:414] "containerMap: removing container" podUID="20f715b5-f7d0-440c-9401-dc9cf82944e9" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763612    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f715b5-f7d0-440c-9401-dc9cf82944e9" containerName="maas-api"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763619    5906 cpu_manager.go:414] "containerMap: removing container" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763625    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763632    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763637    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763643    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763648    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763655    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763660    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy"
Oct 31 04:10:36.763654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763669    5906 cpu_manager.go:414] "containerMap: removing container" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763674    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763682    5906 cpu_manager.go:414] "containerMap: removing container" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763687    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763693    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763698    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763704    5906 cpu_manager.go:414] "containerMap: removing container" podUID="48c9ffad-9df1-429a-b483-170065a30abb" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763708    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48c9ffad-9df1-429a-b483-170065a30abb" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763715    5906 cpu_manager.go:414] "containerMap: removing container" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763719    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763731    5906 cpu_manager.go:414] "containerMap: removing container" podUID="d99c7b2d-bfad-45c9-9691-6e888fab2082" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763736    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d99c7b2d-bfad-45c9-9691-6e888fab2082" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763744    5906 cpu_manager.go:414] "containerMap: removing container" podUID="9a787113-91b9-4814-8828-501ac0bb5f76" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763749    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a787113-91b9-4814-8828-501ac0bb5f76" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763756    5906 cpu_manager.go:414] "containerMap: removing container" podUID="dd808397-af2b-4790-9325-b936b3a052de" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763761    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd808397-af2b-4790-9325-b936b3a052de" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763769    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763773    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763780    5906 cpu_manager.go:414] "containerMap: removing container" podUID="2741d67f-f693-4bb4-99fe-c169b380e663" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763785    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2741d67f-f693-4bb4-99fe-c169b380e663" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763846    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="169384ae-a553-458b-b56b-7c311ab17d38" containerName="main"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763856    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f91b7ef-f035-4f81-951d-9dc17c538592" containerName="main"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763862    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763867    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="521f908c-8733-4b71-af51-4f0fcd339c09" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763874    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e944735-7b85-4160-a3ec-a4cefd28f746" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763881    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="02ca1b2d-2f44-47ed-9ccc-99bcc5285e7e" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763886    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763891    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a79b0003-c031-4b13-9375-97a1d0ccefbb" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763896    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc17dd46-e662-49bc-8994-a73af37493a3" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763902    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="57f0f9df-fecb-415f-8cac-441031401154" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763908    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2dccd3f-32eb-4725-8957-7cd1d61fe22a" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763913    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a41ca42-cd82-4aaa-b6f7-33c2df136fdb" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763919    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f715b5-f7d0-440c-9401-dc9cf82944e9" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763924    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763930    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7064811-4b4b-4608-aa2a-5469c6d2571f" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763934    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763938    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c44d273e-2eb6-41a7-a91e-35905e76ab88" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763944    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="8bfddfc3-1b1a-4166-b8da-181cfd6538cb" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763949    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763954    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1828d0f0-cdbc-4f98-a047-a6ecb71d0a69" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763959    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="48c9ffad-9df1-429a-b483-170065a30abb" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763964    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e688e64-20e8-471d-97b5-55ebadcaa984" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763970    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="d99c7b2d-bfad-45c9-9691-6e888fab2082" containerName="maas-api"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763974    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763980    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a787113-91b9-4814-8828-501ac0bb5f76" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763986    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd808397-af2b-4790-9325-b936b3a052de" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763991    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7b2fc3a-dc93-437c-8cfc-f7727595709c" containerName="payload-pre-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.763997    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="2741d67f-f693-4bb4-99fe-c169b380e663" containerName="payload-processing"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.764004    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.764009    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy"
Oct 31 04:10:36.764650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.764014    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e2ce747-c974-460c-b44d-c4e6f0c62734" containerName="maas-api"
Oct 31 04:10:36.766964 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.766948    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.822833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.822793    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"]
Oct 31 04:10:36.872331 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.872294    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6e81aea5-serving-cert\""
Oct 31 04:10:36.877459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.877419    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.877643 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.877563    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.877745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.877687    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6dj5\" (UniqueName: \"kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.877895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.877863    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.940993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.940957    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:36.978860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.978825    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.979078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.978916    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.979078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.978965    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6dj5\" (UniqueName: \"kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.979078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.978997    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.979078 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:36.979004    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-serving-cert: secret "maas-api-e2e-mt-6e81aea5-serving-cert" not found
Oct 31 04:10:36.979377 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:36.979102    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls podName:ba81e821-2384-48c5-80da-68778851789e nodeName:}" failed. No retries permitted until 2026-10-31 04:10:37.479077452 +0000 UTC m=+2487.312937749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls") pod "maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" (UID: "ba81e821-2384-48c5-80da-68778851789e") : secret "maas-api-e2e-mt-6e81aea5-serving-cert" not found
Oct 31 04:10:36.979701 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.979683    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:36.981584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:36.981563    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:37.045717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:37.045621    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6dj5\" (UniqueName: \"kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:37.089642 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:37.089607    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:37.239047 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:37.239006    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:37.490050 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:37.489932    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:37.492436 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:37.492410    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\") pod \"maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:37.681077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:37.681039    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:38.054285 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:10:38.054250    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba81e821_2384_48c5_80da_68778851789e.slice/crio-0b7c09932d06e82a9093b70fab98d1efbe62f248460b4d3d52251960159549a9 WatchSource:0}: Error finding container 0b7c09932d06e82a9093b70fab98d1efbe62f248460b4d3d52251960159549a9: Status 404 returned error can't find the container with id 0b7c09932d06e82a9093b70fab98d1efbe62f248460b4d3d52251960159549a9
Oct 31 04:10:38.055770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.055745    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"]
Oct 31 04:10:38.056314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.056293    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:10:38.271932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.271885    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:38.338004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.337931    5906 patch_prober.go:29] interesting pod/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:10:38.338004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.337981    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:10:38.419410 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.419373    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:38.712694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:38.712605    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" event={"ID":"ba81e821-2384-48c5-80da-68778851789e","Type":"ContainerStarted","Data":"0b7c09932d06e82a9093b70fab98d1efbe62f248460b4d3d52251960159549a9"}
Oct 31 04:10:39.719769 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:39.719728    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" event={"ID":"ba81e821-2384-48c5-80da-68778851789e","Type":"ContainerStarted","Data":"a181c016bd0d0704fe1468a1409329dbdc723227fd677b04cc09061fc1254ae9"}
Oct 31 04:10:39.720260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:39.719915    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:40.041584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.041553    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 04:10:40.092070 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.092038    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:40.138018 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.137984    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138045    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138099    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138128    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138155    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138183    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138283    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138399    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138464    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data\") pod \"ae211c5c-03d3-4978-85ec-65f045d3d318\" (UID: \"ae211c5c-03d3-4978-85ec-65f045d3d318\") "
Oct 31 04:10:40.138557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138479    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:40.138557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138492    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:40.138557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138498    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:40.138862 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138791    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:40.138862 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.138828    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:40.139143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.139099    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.139143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.139125    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.139143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.139141    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.139431 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.139158    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.139431 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.139172    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae211c5c-03d3-4978-85ec-65f045d3d318-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.140363 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.140338    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:40.140611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.140590    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:40.140695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.140613    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:10:40.140695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.140625    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw" pod "ae211c5c-03d3-4978-85ec-65f045d3d318" (UID: "ae211c5c-03d3-4978-85ec-65f045d3d318"). InnerVolumeSpecName "kube-api-access-bh6mw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:40.165859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.165808    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" podStartSLOduration=3.170288817 podStartE2EDuration="4.165793845s" totalImagesPullingTime="995.505028ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:10:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:10:39.849345167 +0000 UTC m=+2489.683205485" watchObservedRunningTime="2026-10-31 04:10:40.165793845 +0000 UTC m=+2489.999654186"
Oct 31 04:10:40.240614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.240578    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh6mw\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-kube-api-access-bh6mw\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.240614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.240614    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.240837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.240627    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.240837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.240639    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae211c5c-03d3-4978-85ec-65f045d3d318-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:40.726306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.726199    5906 generic.go:356] "Generic (PLEG): container finished" podID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerID="6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996" exitCode=0
Oct 31 04:10:40.726306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.726250    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" event={"ID":"ae211c5c-03d3-4978-85ec-65f045d3d318","Type":"ContainerDied","Data":"6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996"}
Oct 31 04:10:40.726306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.726280    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"
Oct 31 04:10:40.726306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.726306    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647" event={"ID":"ae211c5c-03d3-4978-85ec-65f045d3d318","Type":"ContainerDied","Data":"6f0ef310a44c5f95ed6d2af8a68b4fc74b855531c1f8ce4c259d87c25cd7d68c"}
Oct 31 04:10:40.726894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.726328    5906 scope.go:122] "RemoveContainer" containerID="6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996"
Oct 31 04:10:40.735888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.735861    5906 scope.go:122] "RemoveContainer" containerID="78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"
Oct 31 04:10:40.745527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.745503    5906 scope.go:122] "RemoveContainer" containerID="6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996"
Oct 31 04:10:40.745873 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:40.745846    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996\": container with ID starting with 6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996 not found: ID does not exist" containerID="6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996"
Oct 31 04:10:40.745965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.745884    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996"} err="failed to get container status \"6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996\": rpc error: code = NotFound desc = could not find container \"6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996\": container with ID starting with 6bef6f1abe613f36a1b1b52bb78d3d69f6b9a46e29d1eb023d4cc6c79b92c996 not found: ID does not exist"
Oct 31 04:10:40.745965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.745913    5906 scope.go:122] "RemoveContainer" containerID="78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"
Oct 31 04:10:40.746164 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:40.746144    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac\": container with ID starting with 78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac not found: ID does not exist" containerID="78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"
Oct 31 04:10:40.746238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.746172    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac"} err="failed to get container status \"78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac\": rpc error: code = NotFound desc = could not find container \"78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac\": container with ID starting with 78d97df9d2ee3c08b628195963de357f1a7d0417d0adb77e5d698523c771bbac not found: ID does not exist"
Oct 31 04:10:40.902091 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.902050    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"]
Oct 31 04:10:40.935984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:40.935940    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w5-938507-openshift-default-66b5df7b65-zb647"]
Oct 31 04:10:41.903569 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:41.903532    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:42.351814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:42.351777    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" path="/var/lib/kubelet/pods/ae211c5c-03d3-4978-85ec-65f045d3d318/volumes"
Oct 31 04:10:45.732247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:45.732219    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:10:46.115314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:46.115270    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:46.257061 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:46.256993    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:46.499866 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:46.499836    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:46.645906 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:46.645877    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:47.404860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:47.404826    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:47.543327 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:47.543291    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:50.344389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:50.344356    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:51.328633 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.328588    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"]
Oct 31 04:10:51.329353 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.328866    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" podUID="bdfe7afd-4334-4df4-99c0-7cc0579eddc8" containerName="maas-api" containerID="cri-o://94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb" gracePeriod=30
Oct 31 04:10:51.419645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.419616    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:51.637260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.637227    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:10:51.763476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.763443    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:51.814060 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.814021    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs\") pod \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") "
Oct 31 04:10:51.814300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.814094    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw\" (UniqueName: \"kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw\") pod \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") "
Oct 31 04:10:51.814300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.814130    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca\") pod \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") "
Oct 31 04:10:51.814300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.814230    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\") pod \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\" (UID: \"bdfe7afd-4334-4df4-99c0-7cc0579eddc8\") "
Oct 31 04:10:51.814552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.814519    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca" pod "bdfe7afd-4334-4df4-99c0-7cc0579eddc8" (UID: "bdfe7afd-4334-4df4-99c0-7cc0579eddc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:51.814873 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.814845    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:51.815956 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.815926    5906 generic.go:356] "Generic (PLEG): container finished" podID="bdfe7afd-4334-4df4-99c0-7cc0579eddc8" containerID="94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb" exitCode=0
Oct 31 04:10:51.816094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816018    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" event={"ID":"bdfe7afd-4334-4df4-99c0-7cc0579eddc8","Type":"ContainerDied","Data":"94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb"}
Oct 31 04:10:51.816094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816046    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"
Oct 31 04:10:51.816094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816076    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg" event={"ID":"bdfe7afd-4334-4df4-99c0-7cc0579eddc8","Type":"ContainerDied","Data":"d8dd283027c8219eaa4b5d29ebe0c27562bb36f71f1c1ea623c3bf2ccb2873d2"}
Oct 31 04:10:51.816737 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816110    5906 scope.go:122] "RemoveContainer" containerID="94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb"
Oct 31 04:10:51.816942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816903    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw" pod "bdfe7afd-4334-4df4-99c0-7cc0579eddc8" (UID: "bdfe7afd-4334-4df4-99c0-7cc0579eddc8"). InnerVolumeSpecName "kube-api-access-g5bbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:51.816942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816915    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs" pod "bdfe7afd-4334-4df4-99c0-7cc0579eddc8" (UID: "bdfe7afd-4334-4df4-99c0-7cc0579eddc8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:51.817078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.816995    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls" pod "bdfe7afd-4334-4df4-99c0-7cc0579eddc8" (UID: "bdfe7afd-4334-4df4-99c0-7cc0579eddc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:51.840605 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.840579    5906 scope.go:122] "RemoveContainer" containerID="94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb"
Oct 31 04:10:51.840929 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:10:51.840893    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb\": container with ID starting with 94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb not found: ID does not exist" containerID="94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb"
Oct 31 04:10:51.841046 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.840934    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb"} err="failed to get container status \"94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb\": rpc error: code = NotFound desc = could not find container \"94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb\": container with ID starting with 94aea248fe8201e64eecf2d43859d401c8befe65660e6969af81db3a3d7953fb not found: ID does not exist"
Oct 31 04:10:51.915851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.915769    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:51.915851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.915800    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5bbw\" (UniqueName: \"kubernetes.io/projected/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-kube-api-access-g5bbw\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:51.915851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:51.915814    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdfe7afd-4334-4df4-99c0-7cc0579eddc8-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:52.046706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:52.046671    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:52.248226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:52.247854    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"]
Oct 31 04:10:52.250597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:52.250549    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w5-440092-65d95c9c79-xhjdg"]
Oct 31 04:10:52.351607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:52.351570    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdfe7afd-4334-4df4-99c0-7cc0579eddc8" path="/var/lib/kubelet/pods/bdfe7afd-4334-4df4-99c0-7cc0579eddc8/volumes"
Oct 31 04:10:52.685611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:52.685525    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:52.983164 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:52.983065    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:53.239070 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:53.238972    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:53.380165 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:53.380133    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:53.978082 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:53.978044    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:58.221945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:58.221903    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:59.033166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:59.033131    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:59.214870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:59.214832    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:10:59.367300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:59.367166    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"]
Oct 31 04:10:59.367839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:59.367701    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy" containerID="cri-o://5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953" gracePeriod=30
Oct 31 04:10:59.368480 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:10:59.368426    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:00.593220 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:00.593183    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:00.735835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:00.735800    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:02.700943 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:02.700905    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:04.633303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.633230    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 04:11:04.793856 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.793815    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794044 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.793946    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794044 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794000    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794044 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794025    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794073    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794113    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794145    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794263    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794318    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert\") pod \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\" (UID: \"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2\") "
Oct 31 04:11:04.794487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794348    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.794588 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794481    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.794588 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794559    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.794691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794664    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.794873 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794851    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:04.795011 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.794868    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.795125 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.795107    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.795241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.795226    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.795355 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.795342    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.797103 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.797049    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:04.797350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.797325    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:04.797444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.797365    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.797592 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.797567    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m" pod "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" (UID: "19f972ea-4c3b-4a95-8b59-32cefbbe9ac2"). InnerVolumeSpecName "kube-api-access-wqk5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:04.896691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.896593    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.896691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.896621    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.896691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.896630    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.896691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.896639    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.896691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.896648    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqk5m\" (UniqueName: \"kubernetes.io/projected/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2-kube-api-access-wqk5m\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.900399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.900365    5906 generic.go:356] "Generic (PLEG): container finished" podID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerID="5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953" exitCode=0
Oct 31 04:11:04.900566 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.900419    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"
Oct 31 04:11:04.900566 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.900447    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" event={"ID":"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2","Type":"ContainerDied","Data":"5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953"}
Oct 31 04:11:04.900566 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.900509    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf" event={"ID":"19f972ea-4c3b-4a95-8b59-32cefbbe9ac2","Type":"ContainerDied","Data":"185c19f24a9614f08d5f71f101c2623e8fb64e665b976af51b8fe470f9ae247a"}
Oct 31 04:11:04.900566 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.900542    5906 scope.go:122] "RemoveContainer" containerID="5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953"
Oct 31 04:11:04.911612 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.911591    5906 scope.go:122] "RemoveContainer" containerID="16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"
Oct 31 04:11:04.920897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.920870    5906 scope.go:122] "RemoveContainer" containerID="5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953"
Oct 31 04:11:04.921263 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:04.921238    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953\": container with ID starting with 5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953 not found: ID does not exist" containerID="5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953"
Oct 31 04:11:04.921358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.921278    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953"} err="failed to get container status \"5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953\": rpc error: code = NotFound desc = could not find container \"5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953\": container with ID starting with 5d97e883257b867050cc6d84aabe42aeb4b2eaa224fea849cb46a704000aa953 not found: ID does not exist"
Oct 31 04:11:04.921358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.921306    5906 scope.go:122] "RemoveContainer" containerID="16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"
Oct 31 04:11:04.921596 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:04.921576    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3\": container with ID starting with 16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3 not found: ID does not exist" containerID="16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"
Oct 31 04:11:04.921642 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:04.921602    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3"} err="failed to get container status \"16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3\": rpc error: code = NotFound desc = could not find container \"16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3\": container with ID starting with 16ba69f0f85d1adf998779ad89d4fff75fc7f469d0eba6782e7957ff89631bc3 not found: ID does not exist"
Oct 31 04:11:05.079643 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:05.079603    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"]
Oct 31 04:11:05.084618 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:05.084589    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-wl5qf"]
Oct 31 04:11:06.352196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:06.352167    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" path="/var/lib/kubelet/pods/19f972ea-4c3b-4a95-8b59-32cefbbe9ac2/volumes"
Oct 31 04:11:06.381195 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:06.381154    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:11:07.396684 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:07.396647    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:10.196000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:10.195965    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:10.336376 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:10.336337    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:10.922835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:10.922796    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"]
Oct 31 04:11:10.923078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:10.923024    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy" containerID="cri-o://d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65" gracePeriod=30
Oct 31 04:11:10.925158 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:10.925137    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:11.384003 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:11.383969    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:12.032465 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:12.032425    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:16.189320 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.189257    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 04:11:16.236383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236336    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236414    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236458    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236498    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236545    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236769 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236623    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236769 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236723    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.236880 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236833    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:16.236880 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236868    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.237008 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236870    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:16.237008 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236880    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:16.237008 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.236897    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data\") pod \"22ae802c-e968-4325-9415-0ff3be0e670b\" (UID: \"22ae802c-e968-4325-9415-0ff3be0e670b\") "
Oct 31 04:11:16.237173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237070    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:16.237344 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237322    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:16.237511 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237446    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.237511 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237467    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.237511 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237485    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22ae802c-e968-4325-9415-0ff3be0e670b-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.237511 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237499    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.237738 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.237513    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.238832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.238800    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:16.238946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.238847    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:16.238946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.238883    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "kube-api-access-r78mt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:16.239037 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.238970    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token" pod "22ae802c-e968-4325-9415-0ff3be0e670b" (UID: "22ae802c-e968-4325-9415-0ff3be0e670b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:16.338486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.338450    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.338486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.338482    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r78mt\" (UniqueName: \"kubernetes.io/projected/22ae802c-e968-4325-9415-0ff3be0e670b-kube-api-access-r78mt\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.338486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.338492    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22ae802c-e968-4325-9415-0ff3be0e670b-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.338741 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.338501    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22ae802c-e968-4325-9415-0ff3be0e670b-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:16.352810 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.352782    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"]
Oct 31 04:11:16.539428 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.539386    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:16.683098 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.682997    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:16.975415 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.975328    5906 generic.go:356] "Generic (PLEG): container finished" podID="22ae802c-e968-4325-9415-0ff3be0e670b" containerID="d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65" exitCode=0
Oct 31 04:11:16.975415 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.975390    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"
Oct 31 04:11:16.975611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.975419    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" event={"ID":"22ae802c-e968-4325-9415-0ff3be0e670b","Type":"ContainerDied","Data":"d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65"}
Oct 31 04:11:16.975611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.975479    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6" event={"ID":"22ae802c-e968-4325-9415-0ff3be0e670b","Type":"ContainerDied","Data":"a7642c4847ce698e8dc1579d5ee41cccf38bb5f54cfd1b99a16ce7fed60883f2"}
Oct 31 04:11:16.975611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.975504    5906 scope.go:122] "RemoveContainer" containerID="d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65"
Oct 31 04:11:16.985614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.985590    5906 scope.go:122] "RemoveContainer" containerID="f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"
Oct 31 04:11:16.995154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.995127    5906 scope.go:122] "RemoveContainer" containerID="d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65"
Oct 31 04:11:16.995532 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:16.995505    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65\": container with ID starting with d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65 not found: ID does not exist" containerID="d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65"
Oct 31 04:11:16.995617 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.995541    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65"} err="failed to get container status \"d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65\": rpc error: code = NotFound desc = could not find container \"d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65\": container with ID starting with d0a9ee67927733a3109afb2a02021bed57bda187f4e154d73cfb0c45c6227f65 not found: ID does not exist"
Oct 31 04:11:16.995617 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.995560    5906 scope.go:122] "RemoveContainer" containerID="f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"
Oct 31 04:11:16.995862 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:16.995845    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513\": container with ID starting with f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513 not found: ID does not exist" containerID="f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"
Oct 31 04:11:16.995905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:16.995869    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513"} err="failed to get container status \"f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513\": rpc error: code = NotFound desc = could not find container \"f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513\": container with ID starting with f91a7aa00c11e9f16676ba0b559b22e2819cc98790c8afec66518e0e1005d513 not found: ID does not exist"
Oct 31 04:11:17.135056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:17.135015    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:17.145982 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:17.145946    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"]
Oct 31 04:11:17.200917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:17.200872    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w5-440092-openshift-default-7bcfc67d6f-nvbl6"]
Oct 31 04:11:18.352583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:18.352546    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" path="/var/lib/kubelet/pods/22ae802c-e968-4325-9415-0ff3be0e670b/volumes"
Oct 31 04:11:21.774191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:21.774158    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:23.299601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:23.299567    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"]
Oct 31 04:11:23.641596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:23.641513    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:23.834770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:23.834735    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:25.945264 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:25.945216    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:26.083328 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:26.083281    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:26.860941 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:26.860910    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:28.300102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:28.300056    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerName="payload-processing" containerID="cri-o://4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6" gracePeriod=25
Oct 31 04:11:28.427613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:28.427578    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:30.776817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.776771    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"]
Oct 31 04:11:30.777416 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.777021    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy" containerID="cri-o://aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044" gracePeriod=30
Oct 31 04:11:30.828586 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.828550    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"]
Oct 31 04:11:30.828811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.828773    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" containerID="cri-o://98b7519e854771c0942b5522ed15de37d8cc6c098727253d47f7fba9c297c40f" gracePeriod=30
Oct 31 04:11:30.830132 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.830101    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"]
Oct 31 04:11:30.830317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.830294    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" containerID="cri-o://e01deaab1867b80d861aeb0e48a869d0a6cf9370e6a17ec3bfbfe77214555a68" gracePeriod=30
Oct 31 04:11:30.881384 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.881347    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"]
Oct 31 04:11:30.881615 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.881590    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy" containerID="cri-o://3bbdf96632f0ec69161df258dd28854bc5bbfa150719be17332e2232275553cf" gracePeriod=30
Oct 31 04:11:30.900913 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.900875    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"]
Oct 31 04:11:30.901375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.901342    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" podUID="ba81e821-2384-48c5-80da-68778851789e" containerName="maas-api" containerID="cri-o://a181c016bd0d0704fe1468a1409329dbdc723227fd677b04cc09061fc1254ae9" gracePeriod=30
Oct 31 04:11:30.999387 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:30.999342    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:31.087098 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.087052    5906 generic.go:356] "Generic (PLEG): container finished" podID="ba81e821-2384-48c5-80da-68778851789e" containerID="a181c016bd0d0704fe1468a1409329dbdc723227fd677b04cc09061fc1254ae9" exitCode=0
Oct 31 04:11:31.087302 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.087171    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" event={"ID":"ba81e821-2384-48c5-80da-68778851789e","Type":"ContainerDied","Data":"a181c016bd0d0704fe1468a1409329dbdc723227fd677b04cc09061fc1254ae9"}
Oct 31 04:11:31.187285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.187124    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:11:31.327679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.327588    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs\") pod \"ba81e821-2384-48c5-80da-68778851789e\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") "
Oct 31 04:11:31.327679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.327639    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca\") pod \"ba81e821-2384-48c5-80da-68778851789e\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") "
Oct 31 04:11:31.327922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.327735    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\") pod \"ba81e821-2384-48c5-80da-68778851789e\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") "
Oct 31 04:11:31.327922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.327765    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5\" (UniqueName: \"kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5\") pod \"ba81e821-2384-48c5-80da-68778851789e\" (UID: \"ba81e821-2384-48c5-80da-68778851789e\") "
Oct 31 04:11:31.328201 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.328159    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca" pod "ba81e821-2384-48c5-80da-68778851789e" (UID: "ba81e821-2384-48c5-80da-68778851789e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:31.330007 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.329980    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls" pod "ba81e821-2384-48c5-80da-68778851789e" (UID: "ba81e821-2384-48c5-80da-68778851789e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:31.330007 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.330001    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs" pod "ba81e821-2384-48c5-80da-68778851789e" (UID: "ba81e821-2384-48c5-80da-68778851789e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:31.330385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.330365    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5" pod "ba81e821-2384-48c5-80da-68778851789e" (UID: "ba81e821-2384-48c5-80da-68778851789e"). InnerVolumeSpecName "kube-api-access-m6dj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:31.374558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.374518    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main" containerID="cri-o://919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739" gracePeriod=45
Oct 31 04:11:31.396263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.396164    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:11:31.430199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.430163    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.430585 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.430496    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba81e821-2384-48c5-80da-68778851789e-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.430585 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.430529    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ba81e821-2384-48c5-80da-68778851789e-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.430585 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.430547    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6dj5\" (UniqueName: \"kubernetes.io/projected/ba81e821-2384-48c5-80da-68778851789e-kube-api-access-m6dj5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.623717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.623688    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 04:11:31.734973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.734932    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs\") pod \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") "
Oct 31 04:11:31.735199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735037    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm\") pod \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") "
Oct 31 04:11:31.735199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735089    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir\") pod \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") "
Oct 31 04:11:31.735199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735123    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache\") pod \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") "
Oct 31 04:11:31.735199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735187    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd\" (UniqueName: \"kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd\") pod \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") "
Oct 31 04:11:31.735483 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735307    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home\") pod \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\" (UID: \"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b\") "
Oct 31 04:11:31.735483 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735445    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache" pod "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" (UID: "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.735633 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735452    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir" pod "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" (UID: "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.735689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735665    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home" pod "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" (UID: "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.735842 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735819    5906 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-home\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.735890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735848    5906 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tmp-dir\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.735890 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.735864    5906 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-model-cache\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.737604 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.737565    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs" pod "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" (UID: "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:31.737730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.737618    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd" pod "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" (UID: "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b"). InnerVolumeSpecName "kube-api-access-4ncbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:31.737730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.737628    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm" pod "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" (UID: "ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.837174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.837140    5906 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-tls-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.837174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.837170    5906 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-dshm\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.837174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:31.837180    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ncbd\" (UniqueName: \"kubernetes.io/projected/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b-kube-api-access-4ncbd\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:32.094287 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.094251    5906 generic.go:356] "Generic (PLEG): container finished" podID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerID="919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739" exitCode=0
Oct 31 04:11:32.094482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.094319    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"
Oct 31 04:11:32.094482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.094340    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" event={"ID":"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b","Type":"ContainerDied","Data":"919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739"}
Oct 31 04:11:32.094482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.094392    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm" event={"ID":"ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b","Type":"ContainerDied","Data":"624bfd01263521695583e9be7f14cf3d416754e3157fa729526e3ba6db203d06"}
Oct 31 04:11:32.094482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.094415    5906 scope.go:122] "RemoveContainer" containerID="919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739"
Oct 31 04:11:32.096027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.096004    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"
Oct 31 04:11:32.096157 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.096005    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27" event={"ID":"ba81e821-2384-48c5-80da-68778851789e","Type":"ContainerDied","Data":"0b7c09932d06e82a9093b70fab98d1efbe62f248460b4d3d52251960159549a9"}
Oct 31 04:11:32.103561 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.103540    5906 scope.go:122] "RemoveContainer" containerID="919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739"
Oct 31 04:11:32.103868 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:32.103846    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739\": container with ID starting with 919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739 not found: ID does not exist" containerID="919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739"
Oct 31 04:11:32.103966 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.103878    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739"} err="failed to get container status \"919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739\": rpc error: code = NotFound desc = could not find container \"919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739\": container with ID starting with 919dc81bf2c814f29f1b9df6fa84bd5e3f414249eedecd553591afc7c4e40739 not found: ID does not exist"
Oct 31 04:11:32.103966 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.103902    5906 scope.go:122] "RemoveContainer" containerID="a181c016bd0d0704fe1468a1409329dbdc723227fd677b04cc09061fc1254ae9"
Oct 31 04:11:32.297754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.297718    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"]
Oct 31 04:11:32.303597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.303571    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6e81aea5-7d79b5557b-bdh27"]
Oct 31 04:11:32.351626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.351583    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba81e821-2384-48c5-80da-68778851789e" path="/var/lib/kubelet/pods/ba81e821-2384-48c5-80da-68778851789e/volumes"
Oct 31 04:11:32.418108 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.418065    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"]
Oct 31 04:11:32.427259 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.427229    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-w5-e5b8f9-kserve-78dd8cfd57-b4ksm"]
Oct 31 04:11:32.481727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.481696    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:32.734939 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.734907    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:32.871924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:32.871891    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:33.377870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:33.377841    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:34.351461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:34.351429    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" path="/var/lib/kubelet/pods/ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b/volumes"
Oct 31 04:11:34.838573 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:34.838531    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-7wm5c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:11:34.838748 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:34.838589    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:11:34.878383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:34.878344    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kl6kq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:11:34.878595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:34.878404    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:11:36.091034 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.090987    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 04:11:36.124412 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.124374    5906 generic.go:356] "Generic (PLEG): container finished" podID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerID="3bbdf96632f0ec69161df258dd28854bc5bbfa150719be17332e2232275553cf" exitCode=0
Oct 31 04:11:36.124608 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.124446    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" event={"ID":"dacb9ec4-66fa-4ed2-b074-285e281e76fb","Type":"ContainerDied","Data":"3bbdf96632f0ec69161df258dd28854bc5bbfa150719be17332e2232275553cf"}
Oct 31 04:11:36.126487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.126464    5906 generic.go:356] "Generic (PLEG): container finished" podID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerID="e01deaab1867b80d861aeb0e48a869d0a6cf9370e6a17ec3bfbfe77214555a68" exitCode=0
Oct 31 04:11:36.126613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.126533    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" event={"ID":"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f","Type":"ContainerDied","Data":"e01deaab1867b80d861aeb0e48a869d0a6cf9370e6a17ec3bfbfe77214555a68"}
Oct 31 04:11:36.128129 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.128103    5906 generic.go:356] "Generic (PLEG): container finished" podID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerID="aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044" exitCode=0
Oct 31 04:11:36.128270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.128156    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"
Oct 31 04:11:36.128270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.128192    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" event={"ID":"2279d7a6-0e60-46c6-96b2-2c702eeb203b","Type":"ContainerDied","Data":"aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044"}
Oct 31 04:11:36.128379 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.128289    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p" event={"ID":"2279d7a6-0e60-46c6-96b2-2c702eeb203b","Type":"ContainerDied","Data":"ccf797723362cd07cfba441e6afc1cefe66b44bc48c1e7f90f3e6dcf159251cc"}
Oct 31 04:11:36.128379 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.128321    5906 scope.go:122] "RemoveContainer" containerID="aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044"
Oct 31 04:11:36.131102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.130997    5906 generic.go:356] "Generic (PLEG): container finished" podID="144777ed-8620-4ddd-99a8-514d12f2510a" containerID="98b7519e854771c0942b5522ed15de37d8cc6c098727253d47f7fba9c297c40f" exitCode=0
Oct 31 04:11:36.131102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.131046    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" event={"ID":"144777ed-8620-4ddd-99a8-514d12f2510a","Type":"ContainerDied","Data":"98b7519e854771c0942b5522ed15de37d8cc6c098727253d47f7fba9c297c40f"}
Oct 31 04:11:36.144292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.144269    5906 scope.go:122] "RemoveContainer" containerID="aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044"
Oct 31 04:11:36.144600 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:36.144578    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044\": container with ID starting with aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044 not found: ID does not exist" containerID="aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044"
Oct 31 04:11:36.144700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.144611    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044"} err="failed to get container status \"aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044\": rpc error: code = NotFound desc = could not find container \"aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044\": container with ID starting with aad083cd7729436defb9d3c5f1a6218f633f3291c821dbd029135294cf943044 not found: ID does not exist"
Oct 31 04:11:36.148838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.148809    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201040    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201095    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201169    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201197    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201278    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201308    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201357    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201397    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.201687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.201452    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr\") pod \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\" (UID: \"2279d7a6-0e60-46c6-96b2-2c702eeb203b\") "
Oct 31 04:11:36.202896 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.202813    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.204295 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.204085    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.204610 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.204545    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.205225 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.205177    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:36.205404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.205377    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.206783 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.206762    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.206991 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.206971    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.208536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.208512    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.209965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.209931    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:36.210570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.210543    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "kube-api-access-qrgwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.211834 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.211804    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token" pod "2279d7a6-0e60-46c6-96b2-2c702eeb203b" (UID: "2279d7a6-0e60-46c6-96b2-2c702eeb203b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.225678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.225659    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 04:11:36.228476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.228456    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 04:11:36.307819 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.307724    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.307819 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.307815    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.307901    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.307944    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308011    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308054    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308088    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308146    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308193    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket\") pod \"144777ed-8620-4ddd-99a8-514d12f2510a\" (UID: \"144777ed-8620-4ddd-99a8-514d12f2510a\") "
Oct 31 04:11:36.308292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308263    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.308505 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308348    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.308560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308516    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.308801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308778    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.308894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308807    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.308894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308829    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.308894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308844    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.308894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308860    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.308894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308800    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.309113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308874    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrgwr\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-kube-api-access-qrgwr\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.309113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308927    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.309113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308931    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:36.309113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308945    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.309113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308961    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.309113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.308975    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2279d7a6-0e60-46c6-96b2-2c702eeb203b-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.310246 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.310224    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:36.310297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.310260    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "kube-api-access-zl5cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.310523 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.310507    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.310656 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.310642    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy" pod "144777ed-8620-4ddd-99a8-514d12f2510a" (UID: "144777ed-8620-4ddd-99a8-514d12f2510a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.389366 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:36.389332    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:11:36.409794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409762    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.409955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409805    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.409955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409829    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.409955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409861    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.409955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409910    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.409955 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409948    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.409993    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.410230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410095    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410123    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410155    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410183    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.410230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410225    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410245    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410246    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410347    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410395    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410415    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410450    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410494    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410594    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410575    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410643    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token\") pod \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\" (UID: \"dacb9ec4-66fa-4ed2-b074-285e281e76fb\") "
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410648    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410653    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410683    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs\") pod \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\" (UID: \"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f\") "
Oct 31 04:11:36.410914 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.410809    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.411311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411042    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.411311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411247    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/144777ed-8620-4ddd-99a8-514d12f2510a-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411273    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411289    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411303    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411316    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411330    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411346    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zl5cb\" (UniqueName: \"kubernetes.io/projected/144777ed-8620-4ddd-99a8-514d12f2510a-kube-api-access-zl5cb\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411361    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411375    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411388    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411402    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411417    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411432    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/144777ed-8620-4ddd-99a8-514d12f2510a-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411446    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.411583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411461    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/144777ed-8620-4ddd-99a8-514d12f2510a-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.412176 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.411603    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:36.413016 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.412988    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.413240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.413186    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "kube-api-access-7jt7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.413421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.413354    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:36.413421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.413360    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.413758 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.413733    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.413837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.413787    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8" pod "dacb9ec4-66fa-4ed2-b074-285e281e76fb" (UID: "dacb9ec4-66fa-4ed2-b074-285e281e76fb"). InnerVolumeSpecName "kube-api-access-pgjd8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.413943 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.413926    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:36.414557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.414535    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token" pod "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" (UID: "a64dedea-79ce-4aab-b35d-01fb3d0a5f5f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.512680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512649    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jt7s\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-kube-api-access-7jt7s\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512677    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512688    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512697    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512706    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512714    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512722    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512731    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgjd8\" (UniqueName: \"kubernetes.io/projected/dacb9ec4-66fa-4ed2-b074-285e281e76fb-kube-api-access-pgjd8\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.512911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.512739    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dacb9ec4-66fa-4ed2-b074-285e281e76fb-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.818190 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.818154    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"]
Oct 31 04:11:36.822447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.822415    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mq99p"]
Oct 31 04:11:36.973334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.973304    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"]
Oct 31 04:11:36.973534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:36.973494    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" podUID="f0f60323-8f9e-41ec-bea7-bdd84b92119c" containerName="maas-api" containerID="cri-o://d78639952624a05ed17a4e7e2f6434b9286c235aaaba22f9d3298da31f9f1e86" gracePeriod=30
Oct 31 04:11:37.098793 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.098760    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:37.138799 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.138768    5906 generic.go:356] "Generic (PLEG): container finished" podID="f0f60323-8f9e-41ec-bea7-bdd84b92119c" containerID="d78639952624a05ed17a4e7e2f6434b9286c235aaaba22f9d3298da31f9f1e86" exitCode=0
Oct 31 04:11:37.138972 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.138847    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" event={"ID":"f0f60323-8f9e-41ec-bea7-bdd84b92119c","Type":"ContainerDied","Data":"d78639952624a05ed17a4e7e2f6434b9286c235aaaba22f9d3298da31f9f1e86"}
Oct 31 04:11:37.140390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.140365    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"
Oct 31 04:11:37.140390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.140376    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c" event={"ID":"a64dedea-79ce-4aab-b35d-01fb3d0a5f5f","Type":"ContainerDied","Data":"b0741c8d06988325b27118e154f4be863c6f060d616282c470f7dbcce4fb3fbe"}
Oct 31 04:11:37.140591 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.140435    5906 scope.go:122] "RemoveContainer" containerID="e01deaab1867b80d861aeb0e48a869d0a6cf9370e6a17ec3bfbfe77214555a68"
Oct 31 04:11:37.144797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.143067    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"
Oct 31 04:11:37.144797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.143068    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq" event={"ID":"144777ed-8620-4ddd-99a8-514d12f2510a","Type":"ContainerDied","Data":"2564e7228f76b7d5a7ca24309797bfb776a190992ddf734af5c45838ff120050"}
Oct 31 04:11:37.145010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.144922    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2" event={"ID":"dacb9ec4-66fa-4ed2-b074-285e281e76fb","Type":"ContainerDied","Data":"8a44a682d5c8658965bc9e417730b61532d6be44b4e3c38ac3e07a97a6892442"}
Oct 31 04:11:37.145010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.144931    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"
Oct 31 04:11:37.154613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.153943    5906 scope.go:122] "RemoveContainer" containerID="98b7519e854771c0942b5522ed15de37d8cc6c098727253d47f7fba9c297c40f"
Oct 31 04:11:37.166919 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.166886    5906 scope.go:122] "RemoveContainer" containerID="3bbdf96632f0ec69161df258dd28854bc5bbfa150719be17332e2232275553cf"
Oct 31 04:11:37.257677 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.257649    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 04:11:37.314900 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.314861    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"]
Oct 31 04:11:37.329628 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.324218    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-cdnk2"]
Oct 31 04:11:37.425450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.425336    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\") pod \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") "
Oct 31 04:11:37.425450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.425396    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs\") pod \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") "
Oct 31 04:11:37.425450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.425441    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca\") pod \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") "
Oct 31 04:11:37.425735 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.425511    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f\" (UniqueName: \"kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f\") pod \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\" (UID: \"f0f60323-8f9e-41ec-bea7-bdd84b92119c\") "
Oct 31 04:11:37.426174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.426144    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca" pod "f0f60323-8f9e-41ec-bea7-bdd84b92119c" (UID: "f0f60323-8f9e-41ec-bea7-bdd84b92119c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:37.427836 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.427806    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs" pod "f0f60323-8f9e-41ec-bea7-bdd84b92119c" (UID: "f0f60323-8f9e-41ec-bea7-bdd84b92119c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:37.427966 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.427829    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls" pod "f0f60323-8f9e-41ec-bea7-bdd84b92119c" (UID: "f0f60323-8f9e-41ec-bea7-bdd84b92119c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:37.427966 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.427907    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f" pod "f0f60323-8f9e-41ec-bea7-bdd84b92119c" (UID: "f0f60323-8f9e-41ec-bea7-bdd84b92119c"). InnerVolumeSpecName "kube-api-access-fpp6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:37.436830 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.435842    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"]
Oct 31 04:11:37.439731 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.439602    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-7wm5c"]
Oct 31 04:11:37.527161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.527116    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:37.527161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.527153    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0f60323-8f9e-41ec-bea7-bdd84b92119c-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:37.527161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.527165    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0f60323-8f9e-41ec-bea7-bdd84b92119c-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:37.527161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.527174    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpp6f\" (UniqueName: \"kubernetes.io/projected/f0f60323-8f9e-41ec-bea7-bdd84b92119c-kube-api-access-fpp6f\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:37.551966 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.551931    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"]
Oct 31 04:11:37.556091 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:37.556063    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kl6kq"]
Oct 31 04:11:38.154147 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.154113    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"
Oct 31 04:11:38.154147 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.154132    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt" event={"ID":"f0f60323-8f9e-41ec-bea7-bdd84b92119c","Type":"ContainerDied","Data":"ba1762ac30420a4a5ea5d386a45befb711a9e5db7eb0322deaf605e79c714362"}
Oct 31 04:11:38.154669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.154194    5906 scope.go:122] "RemoveContainer" containerID="d78639952624a05ed17a4e7e2f6434b9286c235aaaba22f9d3298da31f9f1e86"
Oct 31 04:11:38.324132 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.324092    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"]
Oct 31 04:11:38.329360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.329323    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-e5b8f9-855d759897-52qvt"]
Oct 31 04:11:38.351603 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.351567    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" path="/var/lib/kubelet/pods/144777ed-8620-4ddd-99a8-514d12f2510a/volumes"
Oct 31 04:11:38.352102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.352085    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" path="/var/lib/kubelet/pods/2279d7a6-0e60-46c6-96b2-2c702eeb203b/volumes"
Oct 31 04:11:38.352606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.352590    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" path="/var/lib/kubelet/pods/a64dedea-79ce-4aab-b35d-01fb3d0a5f5f/volumes"
Oct 31 04:11:38.353487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.353466    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" path="/var/lib/kubelet/pods/dacb9ec4-66fa-4ed2-b074-285e281e76fb/volumes"
Oct 31 04:11:38.353910 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.353896    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0f60323-8f9e-41ec-bea7-bdd84b92119c" path="/var/lib/kubelet/pods/f0f60323-8f9e-41ec-bea7-bdd84b92119c/volumes"
Oct 31 04:11:38.437012 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.436930    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:38.574515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.574479    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:38.723087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.723000    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:38.894157 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:38.894123    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:39.419144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:39.419113    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:41.395958 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:41.395922    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:11:43.607201 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:43.607167    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:46.156866 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:46.156829    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:46.311048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:46.311013    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:46.508348 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:46.508298    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"]
Oct 31 04:11:46.508590 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:46.508568    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy" containerID="cri-o://8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4" gracePeriod=30
Oct 31 04:11:46.512977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:46.512937    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:48.106438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.106404    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:48.559071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.559039    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 04:11:48.645066 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.645036    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg\" (UniqueName: \"kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg\") pod \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") "
Oct 31 04:11:48.645252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.645165    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume\") pod \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\" (UID: \"b9d19d73-37f6-4667-87c5-7aa6d8afc700\") "
Oct 31 04:11:48.645548 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.645521    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume" pod "b9d19d73-37f6-4667-87c5-7aa6d8afc700" (UID: "b9d19d73-37f6-4667-87c5-7aa6d8afc700"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:48.647001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.646980    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg" pod "b9d19d73-37f6-4667-87c5-7aa6d8afc700" (UID: "b9d19d73-37f6-4667-87c5-7aa6d8afc700"). InnerVolumeSpecName "kube-api-access-95dxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:48.746376 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.746335    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b9d19d73-37f6-4667-87c5-7aa6d8afc700-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:48.746376 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:48.746371    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95dxg\" (UniqueName: \"kubernetes.io/projected/b9d19d73-37f6-4667-87c5-7aa6d8afc700-kube-api-access-95dxg\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:49.233173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.233092    5906 generic.go:356] "Generic (PLEG): container finished" podID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerID="4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6" exitCode=0
Oct 31 04:11:49.233173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.233143    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"
Oct 31 04:11:49.233645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.233145    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" event={"ID":"b9d19d73-37f6-4667-87c5-7aa6d8afc700","Type":"ContainerDied","Data":"4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6"}
Oct 31 04:11:49.233645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.233271    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn" event={"ID":"b9d19d73-37f6-4667-87c5-7aa6d8afc700","Type":"ContainerDied","Data":"6020b8977b8576b97b6db99fb6c441264d011abfbc3841b73087f9bde7f2ee99"}
Oct 31 04:11:49.233645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.233305    5906 scope.go:122] "RemoveContainer" containerID="4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6"
Oct 31 04:11:49.242386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.242360    5906 scope.go:122] "RemoveContainer" containerID="4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6"
Oct 31 04:11:49.242690 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:49.242661    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6\": container with ID starting with 4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6 not found: ID does not exist" containerID="4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6"
Oct 31 04:11:49.242745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.242704    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6"} err="failed to get container status \"4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6\": rpc error: code = NotFound desc = could not find container \"4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6\": container with ID starting with 4e5dbc5cf4eeef0019a837ebf72264558cca446f12dc84b0c56f2c64e435bfe6 not found: ID does not exist"
Oct 31 04:11:49.403747 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.403705    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"]
Oct 31 04:11:49.455268 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:49.455226    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-e5b8f9-8554755fff-ntvxn"]
Oct 31 04:11:50.351802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:50.351767    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" path="/var/lib/kubelet/pods/b9d19d73-37f6-4667-87c5-7aa6d8afc700/volumes"
Oct 31 04:11:51.768410 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.768383    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 04:11:51.879235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879126    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879182    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879261    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879286    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879345    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879420    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879461    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879529    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879457    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.879626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879578    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.879626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879486    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.879626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879567    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp\") pod \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\" (UID: \"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc\") "
Oct 31 04:11:51.879929 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879784    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.879929 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.879829    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:51.880105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.880082    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.880173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.880112    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.880173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.880127    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.880173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.880142    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.880173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.880155    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.881666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.881636    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:51.881781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.881663    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "kube-api-access-rkjwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:51.881781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.881707    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.881781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.881724    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token" pod "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" (UID: "d5d35492-ee24-44d7-8e6b-2db34f6ac5fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:51.980779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.980741    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.980779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.980778    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.980969 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.980791    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.980969 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:51.980800    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkjwp\" (UniqueName: \"kubernetes.io/projected/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc-kube-api-access-rkjwp\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:52.250597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.250560    5906 generic.go:356] "Generic (PLEG): container finished" podID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerID="8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4" exitCode=0
Oct 31 04:11:52.250814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.250607    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"
Oct 31 04:11:52.250814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.250637    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" event={"ID":"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc","Type":"ContainerDied","Data":"8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4"}
Oct 31 04:11:52.250814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.250695    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr" event={"ID":"d5d35492-ee24-44d7-8e6b-2db34f6ac5fc","Type":"ContainerDied","Data":"0d71c7150275dcc141eb885c78577140596c694739954966bc28a290ce6daaa2"}
Oct 31 04:11:52.250814 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.250718    5906 scope.go:122] "RemoveContainer" containerID="8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4"
Oct 31 04:11:52.260429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.260382    5906 scope.go:122] "RemoveContainer" containerID="69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"
Oct 31 04:11:52.269181 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.269157    5906 scope.go:122] "RemoveContainer" containerID="8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4"
Oct 31 04:11:52.269486 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:52.269466    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4\": container with ID starting with 8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4 not found: ID does not exist" containerID="8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4"
Oct 31 04:11:52.269569 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.269494    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4"} err="failed to get container status \"8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4\": rpc error: code = NotFound desc = could not find container \"8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4\": container with ID starting with 8fd7ccf4566530bf17b7d70d51223943b7c3be3efd6218663efba0aa6a98f1d4 not found: ID does not exist"
Oct 31 04:11:52.269569 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.269513    5906 scope.go:122] "RemoveContainer" containerID="69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"
Oct 31 04:11:52.269756 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:11:52.269743    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca\": container with ID starting with 69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca not found: ID does not exist" containerID="69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"
Oct 31 04:11:52.269808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.269759    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca"} err="failed to get container status \"69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca\": rpc error: code = NotFound desc = could not find container \"69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca\": container with ID starting with 69b39b65fe63f0289a15a009a569af7e15064a34018c1574b9760f1f514f95ca not found: ID does not exist"
Oct 31 04:11:52.421021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.420986    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"]
Oct 31 04:11:52.472514 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.472477    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-t9qnr"]
Oct 31 04:11:52.703346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:52.703258    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:54.194431 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:54.194398    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:54.340463 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:54.340430    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:11:54.353041 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:54.352979    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" path="/var/lib/kubelet/pods/d5d35492-ee24-44d7-8e6b-2db34f6ac5fc/volumes"
Oct 31 04:11:57.190993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:11:57.190951    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:00.082843 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:00.082803    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:01.697921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:01.697890    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:06.195004 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:06.194942    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:06.380375 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:12:06.380335    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:12:07.503395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:07.503364    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:10.694223 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:10.694177    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:10.831846 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:10.831813    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:14.628982 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:14.628943    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:15.129891 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:15.129858    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:15.874439 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:15.874404    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:19.704594 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:19.704555    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:22.067050 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:22.067006    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:22.205183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:22.205133    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:22.535989 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:22.535949    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"]
Oct 31 04:12:22.536258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:22.536198    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy" containerID="cri-o://71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6" gracePeriod=30
Oct 31 04:12:22.589454 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:22.589421    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:22.977316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:22.977190    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:23.463875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.463838    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"]
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464285    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f0f60323-8f9e-41ec-bea7-bdd84b92119c" containerName="maas-api"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464300    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0f60323-8f9e-41ec-bea7-bdd84b92119c" containerName="maas-api"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464313    5906 cpu_manager.go:414] "containerMap: removing container" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464318    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464326    5906 cpu_manager.go:414] "containerMap: removing container" podUID="ba81e821-2384-48c5-80da-68778851789e" containerName="maas-api"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464331    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba81e821-2384-48c5-80da-68778851789e" containerName="maas-api"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464338    5906 cpu_manager.go:414] "containerMap: removing container" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464343    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464351    5906 cpu_manager.go:414] "containerMap: removing container" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464356    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464363    5906 cpu_manager.go:414] "containerMap: removing container" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464368    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b2a7874-12f4-49f8-8460-2932d97bf620" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464377    5906 cpu_manager.go:414] "containerMap: removing container" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464382    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464390    5906 cpu_manager.go:414] "containerMap: removing container" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464395    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464403    5906 cpu_manager.go:414] "containerMap: removing container" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464408    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464420    5906 cpu_manager.go:414] "containerMap: removing container" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy"
Oct 31 04:12:23.464417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464425    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464432    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464437    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464444    5906 cpu_manager.go:414] "containerMap: removing container" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464448    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464462    5906 cpu_manager.go:414] "containerMap: removing container" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464467    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2246c16-a828-4f23-9615-06911991656e" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464477    5906 cpu_manager.go:414] "containerMap: removing container" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464481    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464489    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerName="payload-processing"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464493    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerName="payload-processing"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464501    5906 cpu_manager.go:414] "containerMap: removing container" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464505    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464511    5906 cpu_manager.go:414] "containerMap: removing container" podUID="bdfe7afd-4334-4df4-99c0-7cc0579eddc8" containerName="maas-api"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464515    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdfe7afd-4334-4df4-99c0-7cc0579eddc8" containerName="maas-api"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464573    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464581    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464586    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9d19d73-37f6-4667-87c5-7aa6d8afc700" containerName="payload-processing"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464591    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdfe7afd-4334-4df4-99c0-7cc0579eddc8" containerName="maas-api"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464597    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="856a336a-5439-48b5-af95-01572728f411" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464603    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464609    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2691af5-1c21-407e-9627-0ebc83d05b86" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464615    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0f60323-8f9e-41ec-bea7-bdd84b92119c" containerName="maas-api"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464620    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="dacb9ec4-66fa-4ed2-b074-285e281e76fb" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464626    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba81e821-2384-48c5-80da-68778851789e" containerName="maas-api"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464631    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464635    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5d35492-ee24-44d7-8e6b-2db34f6ac5fc" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464642    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="144777ed-8620-4ddd-99a8-514d12f2510a" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464648    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464655    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="2279d7a6-0e60-46c6-96b2-2c702eeb203b" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464661    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a64dedea-79ce-4aab-b35d-01fb3d0a5f5f" containerName="istio-proxy"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464667    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee59a0e5-9bf0-4541-b1ca-b5c6fa6e784b" containerName="main"
Oct 31 04:12:23.465035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.464672    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy"
Oct 31 04:12:23.469671 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.469654    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.509781 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.509737    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mkkn\" (UniqueName: \"kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.509972 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.509889    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.509972 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.509953    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.522144 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.522112    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"]
Oct 31 04:12:23.572385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.572346    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:12:23.572550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.572416    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:12:23.611056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.611004    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.611311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.611065    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.611311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.611151    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkkn\" (UniqueName: \"kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.611809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.611781    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.613518 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.613499    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.673630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.673596    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mkkn\" (UniqueName: \"kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn\") pod \"maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:23.783056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:23.783015    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:24.151331 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:12:24.151293    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c3bbc2_d747_4d72_8da8_2a61d1297522.slice/crio-c10fb860a6542c5d3ad8e1c7739fbd2724ef63af06a1ca8da48bc329ac869a65 WatchSource:0}: Error finding container c10fb860a6542c5d3ad8e1c7739fbd2724ef63af06a1ca8da48bc329ac869a65: Status 404 returned error can't find the container with id c10fb860a6542c5d3ad8e1c7739fbd2724ef63af06a1ca8da48bc329ac869a65
Oct 31 04:12:24.153125 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:24.153093    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"]
Oct 31 04:12:24.414358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:24.414275    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj" event={"ID":"d3c3bbc2-d747-4d72-8da8-2a61d1297522","Type":"ContainerStarted","Data":"c10fb860a6542c5d3ad8e1c7739fbd2724ef63af06a1ca8da48bc329ac869a65"}
Oct 31 04:12:24.432435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:24.432389    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:25.420695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:25.420599    5906 generic.go:356] "Generic (PLEG): container finished" podID="d3c3bbc2-d747-4d72-8da8-2a61d1297522" containerID="d9f86ab6fc877fec89e825535ad80a45cc006c8044d309a7a99480311589aa5e" exitCode=0
Oct 31 04:12:25.420695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:25.420646    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj" event={"ID":"d3c3bbc2-d747-4d72-8da8-2a61d1297522","Type":"ContainerDied","Data":"d9f86ab6fc877fec89e825535ad80a45cc006c8044d309a7a99480311589aa5e"}
Oct 31 04:12:26.562260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.562233    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:26.643324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.643293    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca\") pod \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") "
Oct 31 04:12:26.643507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.643490    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn\" (UniqueName: \"kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn\") pod \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") "
Oct 31 04:12:26.643560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.643523    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca\") pod \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\" (UID: \"d3c3bbc2-d747-4d72-8da8-2a61d1297522\") "
Oct 31 04:12:26.643682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.643656    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca" pod "d3c3bbc2-d747-4d72-8da8-2a61d1297522" (UID: "d3c3bbc2-d747-4d72-8da8-2a61d1297522"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:12:26.643873 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.643856    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:26.645617 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.645590    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn" pod "d3c3bbc2-d747-4d72-8da8-2a61d1297522" (UID: "d3c3bbc2-d747-4d72-8da8-2a61d1297522"). InnerVolumeSpecName "kube-api-access-2mkkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:26.645617 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.645589    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca" pod "d3c3bbc2-d747-4d72-8da8-2a61d1297522" (UID: "d3c3bbc2-d747-4d72-8da8-2a61d1297522"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:26.744685 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.744652    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mkkn\" (UniqueName: \"kubernetes.io/projected/d3c3bbc2-d747-4d72-8da8-2a61d1297522-kube-api-access-2mkkn\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:26.744685 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:26.744683    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3c3bbc2-d747-4d72-8da8-2a61d1297522-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.431357 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.431324    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"
Oct 31 04:12:27.431534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.431359    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj" event={"ID":"d3c3bbc2-d747-4d72-8da8-2a61d1297522","Type":"ContainerDied","Data":"c10fb860a6542c5d3ad8e1c7739fbd2724ef63af06a1ca8da48bc329ac869a65"}
Oct 31 04:12:27.431534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.431423    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c10fb860a6542c5d3ad8e1c7739fbd2724ef63af06a1ca8da48bc329ac869a65"
Oct 31 04:12:27.583950 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:12:27.583910    5906 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab718b0_7c82_4fdf_bae9_4849a5a537e9.slice/crio-conmon-71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab718b0_7c82_4fdf_bae9_4849a5a537e9.slice/crio-71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6.scope\": RecentStats: unable to find data in memory cache]"
Oct 31 04:12:27.800084 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.800059    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:12:27.853347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853314    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853358    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853387    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853431    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853448    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853476    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853513    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853617    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853635    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo\") pod \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\" (UID: \"eab718b0-7c82-4fdf-bae9-4849a5a537e9\") "
Oct 31 04:12:27.853839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853706    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:27.853839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853768    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:12:27.854056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853861    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:27.854056 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.853924    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:27.854166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.854089    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:27.854166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.854107    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.854166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.854130    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.854166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.854145    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.854166 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.854160    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.856017 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.855985    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "kube-api-access-kzxd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:27.856160 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.856051    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:27.856293 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.856273    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:12:27.856362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.856343    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token" pod "eab718b0-7c82-4fdf-bae9-4849a5a537e9" (UID: "eab718b0-7c82-4fdf-bae9-4849a5a537e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:27.955616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.955578    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.955616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.955613    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzxd6\" (UniqueName: \"kubernetes.io/projected/eab718b0-7c82-4fdf-bae9-4849a5a537e9-kube-api-access-kzxd6\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.955616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.955626    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.955875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.955635    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:27.955875 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:27.955644    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eab718b0-7c82-4fdf-bae9-4849a5a537e9-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:28.037535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.037499    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:28.437611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.437574    5906 generic.go:356] "Generic (PLEG): container finished" podID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerID="71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6" exitCode=0
Oct 31 04:12:28.437804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.437623    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"
Oct 31 04:12:28.437804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.437662    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" event={"ID":"eab718b0-7c82-4fdf-bae9-4849a5a537e9","Type":"ContainerDied","Data":"71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6"}
Oct 31 04:12:28.437804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.437718    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4" event={"ID":"eab718b0-7c82-4fdf-bae9-4849a5a537e9","Type":"ContainerDied","Data":"493ecf1013a0955957aa93da9473e149354b66d3876e3037f7080733585b6450"}
Oct 31 04:12:28.437804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.437748    5906 scope.go:122] "RemoveContainer" containerID="71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6"
Oct 31 04:12:28.447195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.447169    5906 scope.go:122] "RemoveContainer" containerID="beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"
Oct 31 04:12:28.456151 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.456122    5906 scope.go:122] "RemoveContainer" containerID="71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6"
Oct 31 04:12:28.456453 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:12:28.456426    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6\": container with ID starting with 71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6 not found: ID does not exist" containerID="71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6"
Oct 31 04:12:28.456520 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.456467    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6"} err="failed to get container status \"71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6\": rpc error: code = NotFound desc = could not find container \"71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6\": container with ID starting with 71a640470b8952baa5ba540e28b37f3b9e19ee9bf5e2746772b78c58ceca8bd6 not found: ID does not exist"
Oct 31 04:12:28.456520 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.456486    5906 scope.go:122] "RemoveContainer" containerID="beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"
Oct 31 04:12:28.456759 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:12:28.456737    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed\": container with ID starting with beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed not found: ID does not exist" containerID="beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"
Oct 31 04:12:28.456804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.456765    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed"} err="failed to get container status \"beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed\": rpc error: code = NotFound desc = could not find container \"beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed\": container with ID starting with beacefaeba2e68029296569e6b55a99b6c4f9525ac1ca3ae18d200ab9599c7ed not found: ID does not exist"
Oct 31 04:12:28.659607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.659526    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"]
Oct 31 04:12:28.662865 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.662838    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6e81aea5-openshift-default-7b856cbcb9-pkqn4"]
Oct 31 04:12:28.760171 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:28.760128    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:29.403833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:29.403800    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:30.351087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:30.351056    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" path="/var/lib/kubelet/pods/eab718b0-7c82-4fdf-bae9-4849a5a537e9/volumes"
Oct 31 04:12:33.983257 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:33.983219    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:34.865923 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:34.865870    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"]
Oct 31 04:12:35.199942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:35.199857    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:35.341321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:35.341286    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:36.381366 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:12:36.381328    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:12:37.941342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941306    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"]
Oct 31 04:12:37.941752 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941735    5906 cpu_manager.go:414] "containerMap: removing container" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy"
Oct 31 04:12:37.941752 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941747    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941761    5906 cpu_manager.go:414] "containerMap: removing container" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941771    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae211c5c-03d3-4978-85ec-65f045d3d318" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941795    5906 cpu_manager.go:414] "containerMap: removing container" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941802    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941811    5906 cpu_manager.go:414] "containerMap: removing container" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941816    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy"
Oct 31 04:12:37.941826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941826    5906 cpu_manager.go:414] "containerMap: removing container" podUID="d3c3bbc2-d747-4d72-8da8-2a61d1297522" containerName="revoke-keys"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941830    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3c3bbc2-d747-4d72-8da8-2a61d1297522" containerName="revoke-keys"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941854    5906 cpu_manager.go:414] "containerMap: removing container" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941859    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19f972ea-4c3b-4a95-8b59-32cefbbe9ac2" containerName="istio-proxy"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941920    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941931    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="eab718b0-7c82-4fdf-bae9-4849a5a537e9" containerName="istio-proxy"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941937    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3c3bbc2-d747-4d72-8da8-2a61d1297522" containerName="revoke-keys"
Oct 31 04:12:37.942048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.941946    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="22ae802c-e968-4325-9415-0ff3be0e670b" containerName="istio-proxy"
Oct 31 04:12:37.944706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:37.944689    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.000252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.000217    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"]
Oct 31 04:12:38.050223 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.050177    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:12:38.050223 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.050184    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:12:38.057156 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.057123    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnpst\" (UniqueName: \"kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.057652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.057428    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.057652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.057481    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.158725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.158687    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnpst\" (UniqueName: \"kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.158938 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.158795    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.158938 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.158836    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.159486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.159459    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.161282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.161261    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.221959 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.221882    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnpst\" (UniqueName: \"kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst\") pod \"maas-api-revoke-sub-52f8d1f7-vchng\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.257586 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.257550    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:38.628115 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:12:38.628081    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6074e71_f4e6_4065_a3bf_6ba7f88d90eb.slice/crio-a01257716dd591ece0046dc4e60c7c20bbdbbf3096710781c1d1d5835ccbb1cd WatchSource:0}: Error finding container a01257716dd591ece0046dc4e60c7c20bbdbbf3096710781c1d1d5835ccbb1cd: Status 404 returned error can't find the container with id a01257716dd591ece0046dc4e60c7c20bbdbbf3096710781c1d1d5835ccbb1cd
Oct 31 04:12:38.629942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:38.629854    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"]
Oct 31 04:12:39.492785 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:39.492747    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng" event={"ID":"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb","Type":"ContainerStarted","Data":"39ab75533347127351c0f1e50e78ccb11d3db703f1d0edb482b6c8e9e5f7b438"}
Oct 31 04:12:39.493180 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:39.492803    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng" event={"ID":"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb","Type":"ContainerStarted","Data":"a01257716dd591ece0046dc4e60c7c20bbdbbf3096710781c1d1d5835ccbb1cd"}
Oct 31 04:12:39.612672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:39.612603    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng" podStartSLOduration=2.6125843680000003 podStartE2EDuration="2.612584368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:12:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:12:39.611367731 +0000 UTC m=+2609.445228048" watchObservedRunningTime="2026-10-31 04:12:39.612584368 +0000 UTC m=+2609.446444671"
Oct 31 04:12:39.867456 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:39.867337    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" podUID="09711f71-2568-419f-8a52-d22a2bd745db" containerName="payload-processing" containerID="cri-o://88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d" gracePeriod=25
Oct 31 04:12:40.498756 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:40.498717    5906 generic.go:356] "Generic (PLEG): container finished" podID="e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" containerID="39ab75533347127351c0f1e50e78ccb11d3db703f1d0edb482b6c8e9e5f7b438" exitCode=0
Oct 31 04:12:40.499177 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:40.498802    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng" event={"ID":"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb","Type":"ContainerDied","Data":"39ab75533347127351c0f1e50e78ccb11d3db703f1d0edb482b6c8e9e5f7b438"}
Oct 31 04:12:41.639827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.639799    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:41.697035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.696999    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst\" (UniqueName: \"kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst\") pod \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") "
Oct 31 04:12:41.697243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.697056    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca\") pod \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") "
Oct 31 04:12:41.697243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.697097    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca\") pod \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\" (UID: \"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb\") "
Oct 31 04:12:41.697512 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.697490    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca" pod "e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" (UID: "e6074e71-f4e6-4065-a3bf-6ba7f88d90eb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:12:41.699200 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.699173    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst" pod "e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" (UID: "e6074e71-f4e6-4065-a3bf-6ba7f88d90eb"). InnerVolumeSpecName "kube-api-access-mnpst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:41.699318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.699198    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca" pod "e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" (UID: "e6074e71-f4e6-4065-a3bf-6ba7f88d90eb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:41.798802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.798709    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnpst\" (UniqueName: \"kubernetes.io/projected/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-kube-api-access-mnpst\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:41.798802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.798745    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:41.798802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:41.798756    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:42.509482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:42.509443    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng" event={"ID":"e6074e71-f4e6-4065-a3bf-6ba7f88d90eb","Type":"ContainerDied","Data":"a01257716dd591ece0046dc4e60c7c20bbdbbf3096710781c1d1d5835ccbb1cd"}
Oct 31 04:12:42.509482 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:42.509481    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"
Oct 31 04:12:42.509706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:42.509500    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a01257716dd591ece0046dc4e60c7c20bbdbbf3096710781c1d1d5835ccbb1cd"
Oct 31 04:12:49.178438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:49.178355    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" podUID="09711f71-2568-419f-8a52-d22a2bd745db" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:12:53.352494 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:53.352461    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:54.538251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:54.538199    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:54.683045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:54.683004    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:54.824184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:54.824101    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:55.059807 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:55.059769    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:55.274637 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:55.274591    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"]
Oct 31 04:12:55.326053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:55.326019    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-412ba5-pk8sj"]
Oct 31 04:12:55.733964 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:55.733881    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:12:56.351861 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:56.351822    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3c3bbc2-d747-4d72-8da8-2a61d1297522" path="/var/lib/kubelet/pods/d3c3bbc2-d747-4d72-8da8-2a61d1297522/volumes"
Oct 31 04:12:59.178288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:12:59.178251    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" podUID="09711f71-2568-419f-8a52-d22a2bd745db" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 31 04:13:00.123414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.123386    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 04:13:00.180897 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.180811    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume\") pod \"09711f71-2568-419f-8a52-d22a2bd745db\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") "
Oct 31 04:13:00.181297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.180909    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5\" (UniqueName: \"kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5\") pod \"09711f71-2568-419f-8a52-d22a2bd745db\" (UID: \"09711f71-2568-419f-8a52-d22a2bd745db\") "
Oct 31 04:13:00.181297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.181172    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume" pod "09711f71-2568-419f-8a52-d22a2bd745db" (UID: "09711f71-2568-419f-8a52-d22a2bd745db"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:13:00.183002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.182978    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5" pod "09711f71-2568-419f-8a52-d22a2bd745db" (UID: "09711f71-2568-419f-8a52-d22a2bd745db"). InnerVolumeSpecName "kube-api-access-x7mx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:13:00.281825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.281790    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7mx5\" (UniqueName: \"kubernetes.io/projected/09711f71-2568-419f-8a52-d22a2bd745db-kube-api-access-x7mx5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:00.281825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.281823    5906 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/09711f71-2568-419f-8a52-d22a2bd745db-plugins-config-volume\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:00.597827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.597787    5906 generic.go:356] "Generic (PLEG): container finished" podID="09711f71-2568-419f-8a52-d22a2bd745db" containerID="88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d" exitCode=0
Oct 31 04:13:00.598014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.597839    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"
Oct 31 04:13:00.598014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.597841    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" event={"ID":"09711f71-2568-419f-8a52-d22a2bd745db","Type":"ContainerDied","Data":"88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d"}
Oct 31 04:13:00.598014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.597964    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm" event={"ID":"09711f71-2568-419f-8a52-d22a2bd745db","Type":"ContainerDied","Data":"327cf1daa437df0d972b4f7e8f404647446812903608857efb75af0775906d32"}
Oct 31 04:13:00.598014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.597994    5906 scope.go:122] "RemoveContainer" containerID="88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d"
Oct 31 04:13:00.606742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.606722    5906 scope.go:122] "RemoveContainer" containerID="88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d"
Oct 31 04:13:00.606996 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:13:00.606975    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d\": container with ID starting with 88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d not found: ID does not exist" containerID="88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d"
Oct 31 04:13:00.607049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.607004    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d"} err="failed to get container status \"88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d\": rpc error: code = NotFound desc = could not find container \"88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d\": container with ID starting with 88f9c5af5fc2b9c8d761986b2de63850084e6e64a8f19487dd505a15d994ed6d not found: ID does not exist"
Oct 31 04:13:00.766233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.766170    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"]
Oct 31 04:13:00.817997 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:00.817967    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-412ba5-5c57b4b689-tdhdm"]
Oct 31 04:13:02.350551 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:02.350514    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09711f71-2568-419f-8a52-d22a2bd745db" path="/var/lib/kubelet/pods/09711f71-2568-419f-8a52-d22a2bd745db/volumes"
Oct 31 04:13:02.564371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:02.564287    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:02.706109 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:02.706011    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:02.924017 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:02.923982    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"]
Oct 31 04:13:02.924288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:02.924242    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy" containerID="cri-o://ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564" gracePeriod=30
Oct 31 04:13:02.930786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:02.930697    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:05.249372 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:05.249332    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:05.389734 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:05.389692    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:05.726099 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:05.726011    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:06.380508 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:13:06.380470    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:13:08.184845 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.184818    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 04:13:08.263944 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.263908    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.263958    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264005    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264042    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264127    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264160    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264233    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264270    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.264421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264320    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:13:08.264986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264950    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket\") pod \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\" (UID: \"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605\") "
Oct 31 04:13:08.265115 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.264979    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:13:08.265184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.265123    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:13:08.265455 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.265418    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:13:08.265591 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.265548    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:13:08.266390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.266365    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.266550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.266393    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.266550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.266416    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.266550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.266433    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.266550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.266452    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.274902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.274195    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "kube-api-access-jnhx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:13:08.274902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.274525    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:13:08.274902 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.274531    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:13:08.275094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.275071    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo" pod "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" (UID: "f9f8a1c0-9eb0-440e-a355-8dbfe08e4605"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:13:08.367659 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.367628    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnhx5\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-kube-api-access-jnhx5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.367659 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.367657    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.367659 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.367666    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.367864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.367674    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:08.635432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.635337    5906 generic.go:356] "Generic (PLEG): container finished" podID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerID="ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564" exitCode=0
Oct 31 04:13:08.635432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.635407    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"
Oct 31 04:13:08.635689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.635428    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" event={"ID":"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605","Type":"ContainerDied","Data":"ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564"}
Oct 31 04:13:08.635689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.635499    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v" event={"ID":"f9f8a1c0-9eb0-440e-a355-8dbfe08e4605","Type":"ContainerDied","Data":"e5dbc92659818705082754863a795e2def6b98f12897f9abe78020aa32ed2718"}
Oct 31 04:13:08.635689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.635529    5906 scope.go:122] "RemoveContainer" containerID="ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564"
Oct 31 04:13:08.644669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.644647    5906 scope.go:122] "RemoveContainer" containerID="ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564"
Oct 31 04:13:08.645004 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:13:08.644974    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564\": container with ID starting with ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564 not found: ID does not exist" containerID="ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564"
Oct 31 04:13:08.645072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.645018    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564"} err="failed to get container status \"ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564\": rpc error: code = NotFound desc = could not find container \"ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564\": container with ID starting with ac878fb6707322b482cd086b444b23d0e8d45d2f2b8b2a6290bb2e30b4b80564 not found: ID does not exist"
Oct 31 04:13:08.805822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.805788    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"]
Oct 31 04:13:08.810079 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:08.810056    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-jhk8v"]
Oct 31 04:13:10.351719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:10.351688    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" path="/var/lib/kubelet/pods/f9f8a1c0-9eb0-440e-a355-8dbfe08e4605/volumes"
Oct 31 04:13:11.791326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:11.791286    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:29.167591 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:29.167549    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:31.575053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575018    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"]
Oct 31 04:13:31.575493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575428    5906 cpu_manager.go:414] "containerMap: removing container" podUID="e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" containerName="revoke-keys"
Oct 31 04:13:31.575493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575439    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" containerName="revoke-keys"
Oct 31 04:13:31.575493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575467    5906 cpu_manager.go:414] "containerMap: removing container" podUID="09711f71-2568-419f-8a52-d22a2bd745db" containerName="payload-processing"
Oct 31 04:13:31.575493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575472    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09711f71-2568-419f-8a52-d22a2bd745db" containerName="payload-processing"
Oct 31 04:13:31.575493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575480    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy"
Oct 31 04:13:31.575493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575484    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy"
Oct 31 04:13:31.575722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575551    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" containerName="revoke-keys"
Oct 31 04:13:31.575722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575564    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="09711f71-2568-419f-8a52-d22a2bd745db" containerName="payload-processing"
Oct 31 04:13:31.575722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.575571    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9f8a1c0-9eb0-440e-a355-8dbfe08e4605" containerName="istio-proxy"
Oct 31 04:13:31.580069 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.580052    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.633934 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.633897    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"]
Oct 31 04:13:31.683486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.683449    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:13:31.683675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.683535    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:13:31.700786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.700751    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f46ct\" (UniqueName: \"kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.700937 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.700881    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.701048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.700950    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.801501 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.801457    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.801730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.801534    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f46ct\" (UniqueName: \"kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.801730 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.801591    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.802173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.802147    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.804048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.804022    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.864823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.864750    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f46ct\" (UniqueName: \"kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct\") pod \"maas-api-revoke-sub-85237e45-jmqsq\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:31.892604 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:31.892570    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:32.281447 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:13:32.281411    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaad4b4df_0d22_49e0_ace2_436a21523927.slice/crio-eaf63b0cb780dcb8d7c05e73613a2954905fc0936c94c00585403e692d1afefb WatchSource:0}: Error finding container eaf63b0cb780dcb8d7c05e73613a2954905fc0936c94c00585403e692d1afefb: Status 404 returned error can't find the container with id eaf63b0cb780dcb8d7c05e73613a2954905fc0936c94c00585403e692d1afefb
Oct 31 04:13:32.282639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:32.282615    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"]
Oct 31 04:13:32.749698 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:32.749663    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq" event={"ID":"aad4b4df-0d22-49e0-ace2-436a21523927","Type":"ContainerStarted","Data":"f3db8e3a39dcda372f62b8ad0b8267b0244606fe42f03b6ff5d810445d5b72fb"}
Oct 31 04:13:32.750162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:32.749715    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq" event={"ID":"aad4b4df-0d22-49e0-ace2-436a21523927","Type":"ContainerStarted","Data":"eaf63b0cb780dcb8d7c05e73613a2954905fc0936c94c00585403e692d1afefb"}
Oct 31 04:13:32.868846 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:32.868740    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq" podStartSLOduration=1.868725975 podStartE2EDuration="1.868725975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:13:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:13:32.868379443 +0000 UTC m=+2662.702239762" watchObservedRunningTime="2026-10-31 04:13:32.868725975 +0000 UTC m=+2662.702586326"
Oct 31 04:13:33.755696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:33.755663    5906 generic.go:356] "Generic (PLEG): container finished" podID="aad4b4df-0d22-49e0-ace2-436a21523927" containerID="f3db8e3a39dcda372f62b8ad0b8267b0244606fe42f03b6ff5d810445d5b72fb" exitCode=0
Oct 31 04:13:33.756119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:33.755750    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq" event={"ID":"aad4b4df-0d22-49e0-ace2-436a21523927","Type":"ContainerDied","Data":"f3db8e3a39dcda372f62b8ad0b8267b0244606fe42f03b6ff5d810445d5b72fb"}
Oct 31 04:13:34.893716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.893687    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:34.930989 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.930947    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca\") pod \"aad4b4df-0d22-49e0-ace2-436a21523927\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") "
Oct 31 04:13:34.931183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.931019    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca\") pod \"aad4b4df-0d22-49e0-ace2-436a21523927\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") "
Oct 31 04:13:34.931183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.931054    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct\" (UniqueName: \"kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct\") pod \"aad4b4df-0d22-49e0-ace2-436a21523927\" (UID: \"aad4b4df-0d22-49e0-ace2-436a21523927\") "
Oct 31 04:13:34.931368 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.931345    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca" pod "aad4b4df-0d22-49e0-ace2-436a21523927" (UID: "aad4b4df-0d22-49e0-ace2-436a21523927"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:13:34.933247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.933196    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct" pod "aad4b4df-0d22-49e0-ace2-436a21523927" (UID: "aad4b4df-0d22-49e0-ace2-436a21523927"). InnerVolumeSpecName "kube-api-access-f46ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:13:34.933247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:34.933196    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca" pod "aad4b4df-0d22-49e0-ace2-436a21523927" (UID: "aad4b4df-0d22-49e0-ace2-436a21523927"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:13:35.032123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:35.032043    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:35.032123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:35.032074    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aad4b4df-0d22-49e0-ace2-436a21523927-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:35.032123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:35.032084    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f46ct\" (UniqueName: \"kubernetes.io/projected/aad4b4df-0d22-49e0-ace2-436a21523927-kube-api-access-f46ct\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:13:35.766943 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:35.766917    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"
Oct 31 04:13:35.767130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:35.766917    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq" event={"ID":"aad4b4df-0d22-49e0-ace2-436a21523927","Type":"ContainerDied","Data":"eaf63b0cb780dcb8d7c05e73613a2954905fc0936c94c00585403e692d1afefb"}
Oct 31 04:13:35.767130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:35.767053    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eaf63b0cb780dcb8d7c05e73613a2954905fc0936c94c00585403e692d1afefb"
Oct 31 04:13:36.381602 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:13:36.381561    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:13:44.016545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:44.016513    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:13:46.807065 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.807028    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"]
Oct 31 04:13:46.807526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.807452    5906 cpu_manager.go:414] "containerMap: removing container" podUID="aad4b4df-0d22-49e0-ace2-436a21523927" containerName="revoke-keys"
Oct 31 04:13:46.807526 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.807465    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aad4b4df-0d22-49e0-ace2-436a21523927" containerName="revoke-keys"
Oct 31 04:13:46.807620 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.807551    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="aad4b4df-0d22-49e0-ace2-436a21523927" containerName="revoke-keys"
Oct 31 04:13:46.810923 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.810895    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835059 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835017    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835073    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8zz2\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835171    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835264    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835297    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835362    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835392    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835483 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835465    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.835522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.835491    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.858939 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.858899    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"]
Oct 31 04:13:46.862567 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.862542    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"]
Oct 31 04:13:46.862700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.862659    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.866682 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.866657    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"]
Oct 31 04:13:46.866796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.866779    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.870496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.870474    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.909246 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.909188    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wk9x4\""
Oct 31 04:13:46.910816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.910786    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"]
Oct 31 04:13:46.936427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936396    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.936427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936431    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.936700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936452    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.936700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936467    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.936700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936484    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.936700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936560    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.936700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936626    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxdl2\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936705    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4k66\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936752    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936800    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936836    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936891    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936907    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936951    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.936983    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbfm\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.936996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937001    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937027    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8zz2\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937065    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937081    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937102    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937123    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937249    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937296    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937362    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937413    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937437    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.937519 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937458    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937535    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937577    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937613    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937690    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937705    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937715    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937730    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937828    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937865    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937903    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937919    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.937942    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.938039    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:46.938122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.938074    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:46.939492 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.939199    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.949569 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.949533    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:46.954193 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.954153    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"]
Oct 31 04:13:46.954530 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.954509    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"]
Oct 31 04:13:46.968935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.968895    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"]
Oct 31 04:13:46.999973 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:46.999936    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:47.000134 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.000109    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8zz2\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2\") pod \"maas-default-gateway-openshift-default-7854949c9-4wtlq\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:47.038965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.038925    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.038965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.038968    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.038997    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.039247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039113    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039149    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039173    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039197    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039265    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039292    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxdl2\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039329    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4k66\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039376    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039403    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039430    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039457    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039507    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039537    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039512    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbfm\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039594    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039596    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039634    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039680    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039724    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039775    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039808    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039870    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039891    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039908    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.039961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039962    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039977    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040027    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040085    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040112    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040191    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040229    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040196    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040294    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040466    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.039826    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.040728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040726    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.041356 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040925    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.041356 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.040959    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.041356 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.041304    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.042530 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.042499    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.042670 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.042647    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.042737 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.042700    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.042888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.042872    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.043219 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.043180    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.044123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.044088    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.103859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.103765    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.103859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.103811    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.103859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.103765    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.103859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.103838    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbfm\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm\") pod \"maas-default-gateway-openshift-default-7854949c9-rx2xb\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.104121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.104040    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxdl2\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2\") pod \"maas-default-gateway-openshift-default-7854949c9-64ln7\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.104121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.104043    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4k66\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66\") pod \"maas-default-gateway-openshift-default-7854949c9-zcwhh\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.124717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.124686    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:47.177414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.177374    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:47.185538 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.185462    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:47.191719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.191683    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:47.505286 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:13:47.505248    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a1ab82b_5c52_4346_8fdf_fbf96210676f.slice/crio-be88bd87a13cf64f878330f611dfd9416330122b526c5fc2b9928ff704b0c5df WatchSource:0}: Error finding container be88bd87a13cf64f878330f611dfd9416330122b526c5fc2b9928ff704b0c5df: Status 404 returned error can't find the container with id be88bd87a13cf64f878330f611dfd9416330122b526c5fc2b9928ff704b0c5df
Oct 31 04:13:47.505952 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.505882    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"]
Oct 31 04:13:47.507624 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.507591    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.507714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.507653    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.507714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.507686    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.602092 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:13:47.601827    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4c1369a_ae0c_476f_b9ea_b31457b9e97b.slice/crio-9137e659b20377e9815eb26b862c513dbcb80efa3f5a2387c924a09d047b93be WatchSource:0}: Error finding container 9137e659b20377e9815eb26b862c513dbcb80efa3f5a2387c924a09d047b93be: Status 404 returned error can't find the container with id 9137e659b20377e9815eb26b862c513dbcb80efa3f5a2387c924a09d047b93be
Oct 31 04:13:47.602267 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.602171    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"]
Oct 31 04:13:47.604374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.604341    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.604492 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.604419    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.604492 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.604463    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.616093 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.615987    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"]
Oct 31 04:13:47.619083 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.619034    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.619185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.619118    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.619185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.619161    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.649352 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.649324    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"]
Oct 31 04:13:47.655397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.652454    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.655397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.652564    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.655397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.652619    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:13:47.827135 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.826964    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" event={"ID":"a4c1369a-ae0c-476f-b9ea-b31457b9e97b","Type":"ContainerStarted","Data":"61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a"}
Oct 31 04:13:47.827135 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.827035    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" event={"ID":"a4c1369a-ae0c-476f-b9ea-b31457b9e97b","Type":"ContainerStarted","Data":"9137e659b20377e9815eb26b862c513dbcb80efa3f5a2387c924a09d047b93be"}
Oct 31 04:13:47.830009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.829925    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" event={"ID":"a5f21edb-6647-45a7-a8db-800cf4748bea","Type":"ContainerStarted","Data":"15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a"}
Oct 31 04:13:47.830009 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.829974    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" event={"ID":"a5f21edb-6647-45a7-a8db-800cf4748bea","Type":"ContainerStarted","Data":"dae4557def4f94462fbc80f87664c0857c4190f6536d99ea57ca8bd6f332c946"}
Oct 31 04:13:47.832827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.832712    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" event={"ID":"67ef6a93-1fe0-44d4-9532-953a9b58dbb1","Type":"ContainerStarted","Data":"dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142"}
Oct 31 04:13:47.832827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.832786    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" event={"ID":"67ef6a93-1fe0-44d4-9532-953a9b58dbb1","Type":"ContainerStarted","Data":"44927d55c580f97e7072f2aa90d05e37f4261fbbeb3a5fa3bbddf6575d0dcfd5"}
Oct 31 04:13:47.835676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.835193    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" event={"ID":"6a1ab82b-5c52-4346-8fdf-fbf96210676f","Type":"ContainerStarted","Data":"ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f"}
Oct 31 04:13:47.835676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.835269    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" event={"ID":"6a1ab82b-5c52-4346-8fdf-fbf96210676f","Type":"ContainerStarted","Data":"be88bd87a13cf64f878330f611dfd9416330122b526c5fc2b9928ff704b0c5df"}
Oct 31 04:13:47.963933 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:47.963869    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podStartSLOduration=1.963849734 podStartE2EDuration="1.963849734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:13:47.962260474 +0000 UTC m=+2677.796120781" watchObservedRunningTime="2026-10-31 04:13:47.963849734 +0000 UTC m=+2677.797710040"
Oct 31 04:13:48.093287 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.092590    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podStartSLOduration=2.092569864 podStartE2EDuration="2.092569864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:13:48.08911074 +0000 UTC m=+2677.922971040" watchObservedRunningTime="2026-10-31 04:13:48.092569864 +0000 UTC m=+2677.926430170"
Oct 31 04:13:48.125534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.125496    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:48.126956 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.126922    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:48.127113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.126986    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:48.178626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.178592    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:48.180371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.180316    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:48.180584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.180559    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:48.188021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.185865    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:48.188021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.187252    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:48.188021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.187299    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:48.192540 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.192514    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:48.196324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.196297    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:48.196515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.196492    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:48.355617 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:48.345900    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podStartSLOduration=2.345836579 podStartE2EDuration="2.345836579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:13:48.218828691 +0000 UTC m=+2678.052688997" watchObservedRunningTime="2026-10-31 04:13:48.345836579 +0000 UTC m=+2678.179696884"
Oct 31 04:13:49.156230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.155701    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:49.156230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.155766    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:49.181229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.181129    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:49.181453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.181228    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:49.205234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.205088    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:49.205234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.205150    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:49.207832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.207793    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:49.208088 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:49.208059    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:50.127230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.126716    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:50.127230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.126785    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:50.182236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.181312    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:50.182236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.181375    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:50.196234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.188493    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:50.196234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.188551    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:50.196234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.192613    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:50.196234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:50.192666    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:51.130229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.129510    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:51.130229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.129585    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:51.180231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.179675    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:51.180231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.179744    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:51.190356 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.190276    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:51.191027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.190950    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:51.195960 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.195931    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:51.196172 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:51.196147    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:52.129233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.126855    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:52.129233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.126919    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:52.180229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.179631    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:52.180229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.179694    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:52.189230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.186903    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:52.189230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.186971    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:52.210229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.209326    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:52.210229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:52.209408    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:53.133241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.132472    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:53.133241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.132552    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:53.184236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.183702    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:53.184236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.183779    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:53.188233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.187530    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:53.188233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.187580    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:53.192636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.192533    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:53.192636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:53.192583    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:54.126231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.125655    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:54.126231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.125715    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:54.180234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.179662    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:54.180234 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.179729    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:54.189229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.187315    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:54.189229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.187373    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:54.199278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.192653    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:54.199278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:54.192729    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:55.126127 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.126082    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 31 04:13:55.126383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.126159    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 31 04:13:55.178059 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.177902    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 31 04:13:55.178059 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.177962    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 31 04:13:55.185910 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.185862    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 31 04:13:55.186107 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.185942    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 31 04:13:55.193184 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.193132    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused" start-of-body=
Oct 31 04:13:55.193422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:55.193222    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": dial tcp 10.134.0.139:15021: connect: connection refused"
Oct 31 04:13:57.201231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.200517    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:57.207229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.202573    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:57.207229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.203818    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:13:57.233359 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.233321    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:13:57.233661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.233635    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:13:57.250435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.250397    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:13:57.250714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.250684    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:13:57.251230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.251192    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:13:57.251406 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.251387    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:13:57.272282 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.272229    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:57.272850 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.272791    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:57.273709 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.273683    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:13:57.299647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.299617    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:57.300108 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.300039    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:57.300942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.300924    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:13:57.328947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.328909    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:57.329537 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.329513    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:57.330312 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.330290    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:13:57.416158 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:13:57.416105    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podStartSLOduration=11.416088726 podStartE2EDuration="11.416088726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:13:48.348594352 +0000 UTC m=+2678.182454661" watchObservedRunningTime="2026-10-31 04:13:57.416088726 +0000 UTC m=+2687.249949031"
Oct 31 04:14:02.069107 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:14:02.069071    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:14:02.217353 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:14:02.217322    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:14:06.381023 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:14:06.380981    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:14:06.454678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:14:06.454622    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:14:29.519729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:14:29.519647    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:14:36.386050 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:14:36.386004    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:15:00.162223 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.162165    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"]
Oct 31 04:15:00.166475 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.166448    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.227247 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.227181    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"]
Oct 31 04:15:00.270195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.270160    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:15:00.302713 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.302664    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890335-cgv5d\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.302936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.302852    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9rvt\" (UniqueName: \"kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt\") pod \"maas-api-key-cleanup-29890335-cgv5d\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.403606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.403561    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9rvt\" (UniqueName: \"kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt\") pod \"maas-api-key-cleanup-29890335-cgv5d\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.403825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.403645    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890335-cgv5d\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.404306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.404282    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890335-cgv5d\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.468014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.467928    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9rvt\" (UniqueName: \"kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt\") pod \"maas-api-key-cleanup-29890335-cgv5d\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.481645 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.481605    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:00.901193 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:15:00.901141    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78eb1107_4533_45f6_a2ee_ac57a4f1a33f.slice/crio-f9fccb1722a3c4f970f4883c42be4440ba19afc1e48aca6f0d9c7708a6b55f22 WatchSource:0}: Error finding container f9fccb1722a3c4f970f4883c42be4440ba19afc1e48aca6f0d9c7708a6b55f22: Status 404 returned error can't find the container with id f9fccb1722a3c4f970f4883c42be4440ba19afc1e48aca6f0d9c7708a6b55f22
Oct 31 04:15:00.917333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:00.917292    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"]
Oct 31 04:15:01.378540 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:01.378496    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" event={"ID":"78eb1107-4533-45f6-a2ee-ac57a4f1a33f","Type":"ContainerStarted","Data":"f9fccb1722a3c4f970f4883c42be4440ba19afc1e48aca6f0d9c7708a6b55f22"}
Oct 31 04:15:02.422300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:02.422180    5906 generic.go:356] "Generic (PLEG): container finished" podID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerID="7105eafa8359acc758335154dd8971af5d9508a2a3dab5680f91ee8e4ebdecf4" exitCode=60
Oct 31 04:15:02.422913 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:02.422876    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" event={"ID":"78eb1107-4533-45f6-a2ee-ac57a4f1a33f","Type":"ContainerDied","Data":"7105eafa8359acc758335154dd8971af5d9508a2a3dab5680f91ee8e4ebdecf4"}
Oct 31 04:15:02.432880 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:02.432849    5906 scope.go:122] "RemoveContainer" containerID="7105eafa8359acc758335154dd8971af5d9508a2a3dab5680f91ee8e4ebdecf4"
Oct 31 04:15:03.430000 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:03.429962    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" event={"ID":"78eb1107-4533-45f6-a2ee-ac57a4f1a33f","Type":"ContainerStarted","Data":"936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9"}
Oct 31 04:15:03.549149 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:03.549092    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" podStartSLOduration=3.549075926 podStartE2EDuration="3.549075926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:15:03.548553065 +0000 UTC m=+2753.382413393" watchObservedRunningTime="2026-10-31 04:15:03.549075926 +0000 UTC m=+2753.382936231"
Oct 31 04:15:04.437679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:04.437643    5906 generic.go:356] "Generic (PLEG): container finished" podID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerID="936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9" exitCode=60
Oct 31 04:15:04.438124 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:04.437734    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" event={"ID":"78eb1107-4533-45f6-a2ee-ac57a4f1a33f","Type":"ContainerDied","Data":"936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9"}
Oct 31 04:15:04.438124 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:04.437791    5906 scope.go:122] "RemoveContainer" containerID="7105eafa8359acc758335154dd8971af5d9508a2a3dab5680f91ee8e4ebdecf4"
Oct 31 04:15:04.438124 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:04.437946    5906 scope.go:122] "RemoveContainer" containerID="936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9"
Oct 31 04:15:04.438272 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:15:04.438249    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890335-cgv5d_odh-ai-gateway-infra(78eb1107-4533-45f6-a2ee-ac57a4f1a33f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f"
Oct 31 04:15:05.444976 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:05.444944    5906 scope.go:122] "RemoveContainer" containerID="936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9"
Oct 31 04:15:05.445477 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:15:05.445240    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890335-cgv5d_odh-ai-gateway-infra(78eb1107-4533-45f6-a2ee-ac57a4f1a33f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f"
Oct 31 04:15:06.379844 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:15:06.379804    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:15:15.345067 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:15.345030    5906 scope.go:122] "RemoveContainer" containerID="936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9"
Oct 31 04:15:16.499793 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:16.499762    5906 generic.go:356] "Generic (PLEG): container finished" podID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerID="958c2f51027480b15585e3e5eafb75db088ca294b16f460a0739e251c9fd4b54" exitCode=60
Oct 31 04:15:16.500260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:16.499839    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" event={"ID":"78eb1107-4533-45f6-a2ee-ac57a4f1a33f","Type":"ContainerDied","Data":"958c2f51027480b15585e3e5eafb75db088ca294b16f460a0739e251c9fd4b54"}
Oct 31 04:15:16.500260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:16.499902    5906 scope.go:122] "RemoveContainer" containerID="936ad439836c858acb6bec6ef117f0dfc6972c5939eb56805dec9d03e5b8bdf9"
Oct 31 04:15:16.500260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:16.500247    5906 scope.go:122] "RemoveContainer" containerID="958c2f51027480b15585e3e5eafb75db088ca294b16f460a0739e251c9fd4b54"
Oct 31 04:15:16.500514 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:15:16.500494    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890335-cgv5d_odh-ai-gateway-infra(78eb1107-4533-45f6-a2ee-ac57a4f1a33f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f"
Oct 31 04:15:17.629707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.629666    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"]
Oct 31 04:15:17.774270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.774242    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:17.870836 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.870798    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca\") pod \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") "
Oct 31 04:15:17.871021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.870874    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt\" (UniqueName: \"kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt\") pod \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\" (UID: \"78eb1107-4533-45f6-a2ee-ac57a4f1a33f\") "
Oct 31 04:15:17.871190 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.871166    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca" pod "78eb1107-4533-45f6-a2ee-ac57a4f1a33f" (UID: "78eb1107-4533-45f6-a2ee-ac57a4f1a33f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:15:17.872993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.872971    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt" pod "78eb1107-4533-45f6-a2ee-ac57a4f1a33f" (UID: "78eb1107-4533-45f6-a2ee-ac57a4f1a33f"). InnerVolumeSpecName "kube-api-access-j9rvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:15:17.971899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.971812    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-maas-api-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:17.971899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:17.971846    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9rvt\" (UniqueName: \"kubernetes.io/projected/78eb1107-4533-45f6-a2ee-ac57a4f1a33f-kube-api-access-j9rvt\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:18.512154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:18.512123    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"
Oct 31 04:15:18.512361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:18.512157    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d" event={"ID":"78eb1107-4533-45f6-a2ee-ac57a4f1a33f","Type":"ContainerDied","Data":"f9fccb1722a3c4f970f4883c42be4440ba19afc1e48aca6f0d9c7708a6b55f22"}
Oct 31 04:15:18.512361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:18.512236    5906 scope.go:122] "RemoveContainer" containerID="958c2f51027480b15585e3e5eafb75db088ca294b16f460a0739e251c9fd4b54"
Oct 31 04:15:18.681794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:18.681709    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"]
Oct 31 04:15:18.732863 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:18.732830    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890335-cgv5d"]
Oct 31 04:15:20.352169 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:20.352133    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" path="/var/lib/kubelet/pods/78eb1107-4533-45f6-a2ee-ac57a4f1a33f/volumes"
Oct 31 04:15:28.250070 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.249323    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:15:28.250070 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.249415    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:15:28.332578 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.332538    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:15:28.332889 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.332855    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:15:28.363372 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.363324    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:15:28.364095 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.364054    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:15:28.378450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.378311    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:15:28.378450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:28.378379    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:15:29.870755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:29.870703    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:15:36.381010 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:15:36.380962    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:15:39.916005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.915964    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"]
Oct 31 04:15:39.916545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.916415    5906 cpu_manager.go:414] "containerMap: removing container" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:39.916545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.916428    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:39.916545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.916461    5906 cpu_manager.go:414] "containerMap: removing container" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:39.916545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.916467    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:39.916545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.916527    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:39.916545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.916539    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:39.919601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.919584    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:39.948222 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.948176    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9dd\" (UniqueName: \"kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:39.948401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.948311    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:39.948401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.948382    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:39.974706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:39.974666    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"]
Oct 31 04:15:40.024185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.024146    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:15:40.024185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.024175    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:15:40.049401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.049360    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.049616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.049501    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9dd\" (UniqueName: \"kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.049616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.049561    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.050052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.050031    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.051947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.051927    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.112788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.112754    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9dd\" (UniqueName: \"kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd\") pod \"maas-api-revoke-sub-2e1f5b5f-78xr2\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.232405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.232321    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:40.602064 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:15:40.602029    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d7dfe2_e358_4fc9_ae0d_c61c4337b319.slice/crio-fbafddb224002e6c1fb815875cf2e97eac9be22b8338b3a1f7e3c20e67d2ced8 WatchSource:0}: Error finding container fbafddb224002e6c1fb815875cf2e97eac9be22b8338b3a1f7e3c20e67d2ced8: Status 404 returned error can't find the container with id fbafddb224002e6c1fb815875cf2e97eac9be22b8338b3a1f7e3c20e67d2ced8
Oct 31 04:15:40.603712 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.603681    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"]
Oct 31 04:15:40.692665 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:40.692628    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2" event={"ID":"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319","Type":"ContainerStarted","Data":"fbafddb224002e6c1fb815875cf2e97eac9be22b8338b3a1f7e3c20e67d2ced8"}
Oct 31 04:15:41.699013 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:41.698919    5906 generic.go:356] "Generic (PLEG): container finished" podID="e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" containerID="32a9800574c324c4a51aa1195ae8df10b2afeae3cc3be39265fec542e88bfab5" exitCode=0
Oct 31 04:15:41.699013 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:41.698998    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2" event={"ID":"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319","Type":"ContainerDied","Data":"32a9800574c324c4a51aa1195ae8df10b2afeae3cc3be39265fec542e88bfab5"}
Oct 31 04:15:42.840035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.840009    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:42.881585 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.881548    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca\") pod \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") "
Oct 31 04:15:42.881766 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.881687    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca\") pod \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") "
Oct 31 04:15:42.881812 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.881770    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd\" (UniqueName: \"kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd\") pod \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\" (UID: \"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319\") "
Oct 31 04:15:42.882024 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.881988    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca" pod "e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" (UID: "e6d7dfe2-e358-4fc9-ae0d-c61c4337b319"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:15:42.883704 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.883673    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca" pod "e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" (UID: "e6d7dfe2-e358-4fc9-ae0d-c61c4337b319"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:15:42.883823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.883769    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd" pod "e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" (UID: "e6d7dfe2-e358-4fc9-ae0d-c61c4337b319"). InnerVolumeSpecName "kube-api-access-cl9dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:15:42.982716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.982619    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl9dd\" (UniqueName: \"kubernetes.io/projected/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-kube-api-access-cl9dd\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:42.982716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.982651    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:42.982716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:42.982661    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:43.712576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:43.712547    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"
Oct 31 04:15:43.712788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:43.712581    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2" event={"ID":"e6d7dfe2-e358-4fc9-ae0d-c61c4337b319","Type":"ContainerDied","Data":"fbafddb224002e6c1fb815875cf2e97eac9be22b8338b3a1f7e3c20e67d2ced8"}
Oct 31 04:15:43.712788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:43.712632    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbafddb224002e6c1fb815875cf2e97eac9be22b8338b3a1f7e3c20e67d2ced8"
Oct 31 04:15:49.743102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:49.743010    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:15:49.888534 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:49.888495    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:15:52.230558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.230518    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"]
Oct 31 04:15:52.231320 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.231297    5906 cpu_manager.go:414] "containerMap: removing container" podUID="e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" containerName="revoke-keys"
Oct 31 04:15:52.231389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.231325    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" containerName="revoke-keys"
Oct 31 04:15:52.231389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.231351    5906 cpu_manager.go:414] "containerMap: removing container" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:52.231389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.231361    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:52.231563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.231548    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" containerName="revoke-keys"
Oct 31 04:15:52.231599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.231568    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="78eb1107-4533-45f6-a2ee-ac57a4f1a33f" containerName="cleanup"
Oct 31 04:15:52.235432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.235403    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.288769 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.288732    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"]
Oct 31 04:15:52.338233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.338190    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:15:52.338429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.338326    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:15:52.382837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.382790    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.383021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.382938    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.383067 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.383042    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsv9k\" (UniqueName: \"kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.484561 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.484464    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.484561 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.484556    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsv9k\" (UniqueName: \"kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.484761 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.484615    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.485354 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.485330    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.486962 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.486940    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.552192 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.552153    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsv9k\" (UniqueName: \"kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k\") pod \"maas-api-revoke-sub-ec843018-8pmgp\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:52.849238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:52.849187    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:53.222820 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:15:53.222776    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30751e83_bd2c_4163_b126_324e96587302.slice/crio-ed53ea63b1f121ef994af1033727b3897c23ef33b124963c88262ae64026e8d0 WatchSource:0}: Error finding container ed53ea63b1f121ef994af1033727b3897c23ef33b124963c88262ae64026e8d0: Status 404 returned error can't find the container with id ed53ea63b1f121ef994af1033727b3897c23ef33b124963c88262ae64026e8d0
Oct 31 04:15:53.224121 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:53.224095    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"]
Oct 31 04:15:53.760560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:53.760517    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp" event={"ID":"30751e83-bd2c-4163-b126-324e96587302","Type":"ContainerStarted","Data":"c63eef5d3b7b0a584345c0b06fa8bdcbc0d35994c4157a1be08846176bb68992"}
Oct 31 04:15:53.760991 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:53.760614    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp" event={"ID":"30751e83-bd2c-4163-b126-324e96587302","Type":"ContainerStarted","Data":"ed53ea63b1f121ef994af1033727b3897c23ef33b124963c88262ae64026e8d0"}
Oct 31 04:15:53.889855 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:53.889729    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp" podStartSLOduration=1.889716596 podStartE2EDuration="1.889716596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:15:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:15:53.889123947 +0000 UTC m=+2803.722984251" watchObservedRunningTime="2026-10-31 04:15:53.889716596 +0000 UTC m=+2803.723576901"
Oct 31 04:15:54.767036 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:54.766991    5906 generic.go:356] "Generic (PLEG): container finished" podID="30751e83-bd2c-4163-b126-324e96587302" containerID="c63eef5d3b7b0a584345c0b06fa8bdcbc0d35994c4157a1be08846176bb68992" exitCode=0
Oct 31 04:15:54.767453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:54.767053    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp" event={"ID":"30751e83-bd2c-4163-b126-324e96587302","Type":"ContainerDied","Data":"c63eef5d3b7b0a584345c0b06fa8bdcbc0d35994c4157a1be08846176bb68992"}
Oct 31 04:15:55.909273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:55.909243    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:56.028755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.028662    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca\") pod \"30751e83-bd2c-4163-b126-324e96587302\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") "
Oct 31 04:15:56.028755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.028704    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k\" (UniqueName: \"kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k\") pod \"30751e83-bd2c-4163-b126-324e96587302\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") "
Oct 31 04:15:56.028755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.028732    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca\") pod \"30751e83-bd2c-4163-b126-324e96587302\" (UID: \"30751e83-bd2c-4163-b126-324e96587302\") "
Oct 31 04:15:56.029074 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.029050    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca" pod "30751e83-bd2c-4163-b126-324e96587302" (UID: "30751e83-bd2c-4163-b126-324e96587302"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:15:56.029295 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.029275    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/30751e83-bd2c-4163-b126-324e96587302-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:56.031499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.031461    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca" pod "30751e83-bd2c-4163-b126-324e96587302" (UID: "30751e83-bd2c-4163-b126-324e96587302"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:15:56.031639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.031602    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k" pod "30751e83-bd2c-4163-b126-324e96587302" (UID: "30751e83-bd2c-4163-b126-324e96587302"). InnerVolumeSpecName "kube-api-access-bsv9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:15:56.130777 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.130732    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsv9k\" (UniqueName: \"kubernetes.io/projected/30751e83-bd2c-4163-b126-324e96587302-kube-api-access-bsv9k\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:56.130777 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.130772    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/30751e83-bd2c-4163-b126-324e96587302-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:15:56.778597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.778569    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"
Oct 31 04:15:56.778597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.778582    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp" event={"ID":"30751e83-bd2c-4163-b126-324e96587302","Type":"ContainerDied","Data":"ed53ea63b1f121ef994af1033727b3897c23ef33b124963c88262ae64026e8d0"}
Oct 31 04:15:56.778816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:15:56.778630    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed53ea63b1f121ef994af1033727b3897c23ef33b124963c88262ae64026e8d0"
Oct 31 04:16:06.381864 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:16:06.381820    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:16:27.636935 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.636900    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"]
Oct 31 04:16:27.637466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.637381    5906 cpu_manager.go:414] "containerMap: removing container" podUID="30751e83-bd2c-4163-b126-324e96587302" containerName="revoke-keys"
Oct 31 04:16:27.637466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.637393    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30751e83-bd2c-4163-b126-324e96587302" containerName="revoke-keys"
Oct 31 04:16:27.637466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.637464    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="30751e83-bd2c-4163-b126-324e96587302" containerName="revoke-keys"
Oct 31 04:16:27.642027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.642004    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.696049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.696015    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"]
Oct 31 04:16:27.724099 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.724063    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvqtw\" (UniqueName: \"kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.724315 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.724118    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.724315 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.724262    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.745022 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.744986    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:16:27.745238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.745082    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:16:27.824970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.824931    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.825183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.825010    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.825183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.825078    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvqtw\" (UniqueName: \"kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.825673 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.825644    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.827355 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.827338    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.888297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.888221    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvqtw\" (UniqueName: \"kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw\") pod \"maas-api-revoke-sub-046e1343-rkdbd\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:27.963341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:27.963306    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:28.336477 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:16:28.336442    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4661cb4_28ca_4060_8caa_419725abe3f3.slice/crio-6e94ff4db008bd7273f2ed3cb66e5864ee8bab7cd590ae7a749a0d30d26d5df1 WatchSource:0}: Error finding container 6e94ff4db008bd7273f2ed3cb66e5864ee8bab7cd590ae7a749a0d30d26d5df1: Status 404 returned error can't find the container with id 6e94ff4db008bd7273f2ed3cb66e5864ee8bab7cd590ae7a749a0d30d26d5df1
Oct 31 04:16:28.338397 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:28.338271    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"]
Oct 31 04:16:28.970691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:28.970601    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd" event={"ID":"a4661cb4-28ca-4060-8caa-419725abe3f3","Type":"ContainerStarted","Data":"ccdc0cd633834ca93e6536e12ff0416a05a3c05c639ce82309a252583b90a55b"}
Oct 31 04:16:28.970691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:28.970656    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd" event={"ID":"a4661cb4-28ca-4060-8caa-419725abe3f3","Type":"ContainerStarted","Data":"6e94ff4db008bd7273f2ed3cb66e5864ee8bab7cd590ae7a749a0d30d26d5df1"}
Oct 31 04:16:29.091242 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:29.091164    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd" podStartSLOduration=2.091148025 podStartE2EDuration="2.091148025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:16:29.089502367 +0000 UTC m=+2838.923362684" watchObservedRunningTime="2026-10-31 04:16:29.091148025 +0000 UTC m=+2838.925008360"
Oct 31 04:16:29.976763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:29.976725    5906 generic.go:356] "Generic (PLEG): container finished" podID="a4661cb4-28ca-4060-8caa-419725abe3f3" containerID="ccdc0cd633834ca93e6536e12ff0416a05a3c05c639ce82309a252583b90a55b" exitCode=0
Oct 31 04:16:29.977240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:29.976813    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd" event={"ID":"a4661cb4-28ca-4060-8caa-419725abe3f3","Type":"ContainerDied","Data":"ccdc0cd633834ca93e6536e12ff0416a05a3c05c639ce82309a252583b90a55b"}
Oct 31 04:16:30.154716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:30.154679    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:16:31.115918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.115894    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:31.157732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.157691    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw\" (UniqueName: \"kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw\") pod \"a4661cb4-28ca-4060-8caa-419725abe3f3\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") "
Oct 31 04:16:31.157908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.157746    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca\") pod \"a4661cb4-28ca-4060-8caa-419725abe3f3\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") "
Oct 31 04:16:31.157908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.157794    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca\") pod \"a4661cb4-28ca-4060-8caa-419725abe3f3\" (UID: \"a4661cb4-28ca-4060-8caa-419725abe3f3\") "
Oct 31 04:16:31.158181 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.158156    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca" pod "a4661cb4-28ca-4060-8caa-419725abe3f3" (UID: "a4661cb4-28ca-4060-8caa-419725abe3f3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:16:31.158288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.158252    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:16:31.159851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.159824    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca" pod "a4661cb4-28ca-4060-8caa-419725abe3f3" (UID: "a4661cb4-28ca-4060-8caa-419725abe3f3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:16:31.159933 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.159902    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw" pod "a4661cb4-28ca-4060-8caa-419725abe3f3" (UID: "a4661cb4-28ca-4060-8caa-419725abe3f3"). InnerVolumeSpecName "kube-api-access-vvqtw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:16:31.259193 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.259156    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvqtw\" (UniqueName: \"kubernetes.io/projected/a4661cb4-28ca-4060-8caa-419725abe3f3-kube-api-access-vvqtw\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:16:31.259193 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.259194    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a4661cb4-28ca-4060-8caa-419725abe3f3-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:16:31.995270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.995237    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"
Oct 31 04:16:31.995453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.995267    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd" event={"ID":"a4661cb4-28ca-4060-8caa-419725abe3f3","Type":"ContainerDied","Data":"6e94ff4db008bd7273f2ed3cb66e5864ee8bab7cd590ae7a749a0d30d26d5df1"}
Oct 31 04:16:31.995453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:31.995352    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e94ff4db008bd7273f2ed3cb66e5864ee8bab7cd590ae7a749a0d30d26d5df1"
Oct 31 04:16:36.381307 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:16:36.381264    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:16:38.344241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.344186    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"]
Oct 31 04:16:38.345093 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.345069    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a4661cb4-28ca-4060-8caa-419725abe3f3" containerName="revoke-keys"
Oct 31 04:16:38.345152 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.345099    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4661cb4-28ca-4060-8caa-419725abe3f3" containerName="revoke-keys"
Oct 31 04:16:38.345280 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.345265    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4661cb4-28ca-4060-8caa-419725abe3f3" containerName="revoke-keys"
Oct 31 04:16:38.349092 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.349069    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.405712 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.405670    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"]
Oct 31 04:16:38.439360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.439322    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4m5t\" (UniqueName: \"kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.439536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.439379    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.439607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.439581    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.453177 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.453108    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:16:38.453394 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.453331    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:16:38.540426 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.540372    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4m5t\" (UniqueName: \"kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.540640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.540463    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.540640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.540592    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.541297 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.541276    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.542852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.542833    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.603536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.603466    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4m5t\" (UniqueName: \"kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t\") pod \"maas-api-revoke-sub-d3fba056-m955j\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:38.662667 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:38.662630    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:39.036015 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:16:39.035985    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87c85cf_848d_4d9a_a19b_f6db38fc7448.slice/crio-4bf02799a7b68c78b962dee537f5eab20b142da3ce10c48f348dd2e1030af60c WatchSource:0}: Error finding container 4bf02799a7b68c78b962dee537f5eab20b142da3ce10c48f348dd2e1030af60c: Status 404 returned error can't find the container with id 4bf02799a7b68c78b962dee537f5eab20b142da3ce10c48f348dd2e1030af60c
Oct 31 04:16:39.037363 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:39.037335    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"]
Oct 31 04:16:40.036838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:40.036804    5906 generic.go:356] "Generic (PLEG): container finished" podID="f87c85cf-848d-4d9a-a19b-f6db38fc7448" containerID="9915249e65fdb1a09400067e42574169ffea560f8d71cbf81e46a766d93493a8" exitCode=0
Oct 31 04:16:40.037258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:40.036893    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j" event={"ID":"f87c85cf-848d-4d9a-a19b-f6db38fc7448","Type":"ContainerDied","Data":"9915249e65fdb1a09400067e42574169ffea560f8d71cbf81e46a766d93493a8"}
Oct 31 04:16:40.037258 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:40.036945    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j" event={"ID":"f87c85cf-848d-4d9a-a19b-f6db38fc7448","Type":"ContainerStarted","Data":"4bf02799a7b68c78b962dee537f5eab20b142da3ce10c48f348dd2e1030af60c"}
Oct 31 04:16:41.244662 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.244626    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:41.375459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.374495    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca\") pod \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") "
Oct 31 04:16:41.375459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.374644    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t\" (UniqueName: \"kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t\") pod \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") "
Oct 31 04:16:41.375459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.374710    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca\") pod \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\" (UID: \"f87c85cf-848d-4d9a-a19b-f6db38fc7448\") "
Oct 31 04:16:41.375459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.375372    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca" pod "f87c85cf-848d-4d9a-a19b-f6db38fc7448" (UID: "f87c85cf-848d-4d9a-a19b-f6db38fc7448"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:16:41.378190 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.378135    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca" pod "f87c85cf-848d-4d9a-a19b-f6db38fc7448" (UID: "f87c85cf-848d-4d9a-a19b-f6db38fc7448"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:16:41.378399 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.378366    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t" pod "f87c85cf-848d-4d9a-a19b-f6db38fc7448" (UID: "f87c85cf-848d-4d9a-a19b-f6db38fc7448"). InnerVolumeSpecName "kube-api-access-q4m5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:16:41.476572 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.476521    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4m5t\" (UniqueName: \"kubernetes.io/projected/f87c85cf-848d-4d9a-a19b-f6db38fc7448-kube-api-access-q4m5t\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:16:41.476572 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.476569    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:16:41.476881 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:41.476587    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87c85cf-848d-4d9a-a19b-f6db38fc7448-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:16:42.081239 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:42.080730    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j" event={"ID":"f87c85cf-848d-4d9a-a19b-f6db38fc7448","Type":"ContainerDied","Data":"4bf02799a7b68c78b962dee537f5eab20b142da3ce10c48f348dd2e1030af60c"}
Oct 31 04:16:42.081239 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:42.080797    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bf02799a7b68c78b962dee537f5eab20b142da3ce10c48f348dd2e1030af60c"
Oct 31 04:16:42.081239 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:42.080865    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"
Oct 31 04:16:43.246228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.245447    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:16:43.246228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.245521    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:16:43.321380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.321334    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:16:43.322641 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.322604    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:16:43.326789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.326759    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:16:43.327070 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.327043    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:16:43.373230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.373056    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:16:43.373230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:16:43.373123    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:17:06.382704 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:17:06.382659    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:17:13.144925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.144894    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"]
Oct 31 04:17:13.145355 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.145339    5906 cpu_manager.go:414] "containerMap: removing container" podUID="f87c85cf-848d-4d9a-a19b-f6db38fc7448" containerName="revoke-keys"
Oct 31 04:17:13.145401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.145356    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f87c85cf-848d-4d9a-a19b-f6db38fc7448" containerName="revoke-keys"
Oct 31 04:17:13.145468 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.145457    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="f87c85cf-848d-4d9a-a19b-f6db38fc7448" containerName="revoke-keys"
Oct 31 04:17:13.148451 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.148429    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.204229 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.204166    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:17:13.236623 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.236589    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.236794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.236637    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld6xg\" (UniqueName: \"kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.236794 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.236717    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.255831 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.255800    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:17:13.257768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.257741    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"]
Oct 31 04:17:13.337350 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.337311    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.337562 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.337363    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld6xg\" (UniqueName: \"kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.337562 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.337386    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.338065 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.338043    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.339746 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.339721    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.402441 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.402368    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld6xg\" (UniqueName: \"kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg\") pod \"maas-api-revoke-sub-ae594979-7n59p\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.463401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.463372    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:13.835168 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:17:13.835131    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31e7199_7dd3_4c92_8796_82be9eefb50e.slice/crio-eb81e3dbc401be8257cac4d22e0266610fa4e7a593d033163852e26a52b7c427 WatchSource:0}: Error finding container eb81e3dbc401be8257cac4d22e0266610fa4e7a593d033163852e26a52b7c427: Status 404 returned error can't find the container with id eb81e3dbc401be8257cac4d22e0266610fa4e7a593d033163852e26a52b7c427
Oct 31 04:17:13.836685 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:13.836657    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"]
Oct 31 04:17:14.295031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:14.294988    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p" event={"ID":"b31e7199-7dd3-4c92-8796-82be9eefb50e","Type":"ContainerStarted","Data":"d3ce5f15e754df0b883f27e085987f20158210483d3121267a5e87386c247a01"}
Oct 31 04:17:14.295438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:14.295049    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p" event={"ID":"b31e7199-7dd3-4c92-8796-82be9eefb50e","Type":"ContainerStarted","Data":"eb81e3dbc401be8257cac4d22e0266610fa4e7a593d033163852e26a52b7c427"}
Oct 31 04:17:14.415284 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:14.415168    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p" podStartSLOduration=1.415155159 podStartE2EDuration="1.415155159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:17:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:17:14.414591028 +0000 UTC m=+2884.248451332" watchObservedRunningTime="2026-10-31 04:17:14.415155159 +0000 UTC m=+2884.249015499"
Oct 31 04:17:15.118983 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:15.118949    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:17:15.263592 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:15.263554    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:17:15.301626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:15.301589    5906 generic.go:356] "Generic (PLEG): container finished" podID="b31e7199-7dd3-4c92-8796-82be9eefb50e" containerID="d3ce5f15e754df0b883f27e085987f20158210483d3121267a5e87386c247a01" exitCode=0
Oct 31 04:17:15.302032 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:15.301677    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p" event={"ID":"b31e7199-7dd3-4c92-8796-82be9eefb50e","Type":"ContainerDied","Data":"d3ce5f15e754df0b883f27e085987f20158210483d3121267a5e87386c247a01"}
Oct 31 04:17:16.448024 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.447999    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:16.571924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.571834    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg\" (UniqueName: \"kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg\") pod \"b31e7199-7dd3-4c92-8796-82be9eefb50e\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") "
Oct 31 04:17:16.571924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.571880    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca\") pod \"b31e7199-7dd3-4c92-8796-82be9eefb50e\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") "
Oct 31 04:17:16.572156 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.571930    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca\") pod \"b31e7199-7dd3-4c92-8796-82be9eefb50e\" (UID: \"b31e7199-7dd3-4c92-8796-82be9eefb50e\") "
Oct 31 04:17:16.572361 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.572325    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca" pod "b31e7199-7dd3-4c92-8796-82be9eefb50e" (UID: "b31e7199-7dd3-4c92-8796-82be9eefb50e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:17:16.574168 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.574134    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg" pod "b31e7199-7dd3-4c92-8796-82be9eefb50e" (UID: "b31e7199-7dd3-4c92-8796-82be9eefb50e"). InnerVolumeSpecName "kube-api-access-ld6xg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:17:16.574311 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.574175    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca" pod "b31e7199-7dd3-4c92-8796-82be9eefb50e" (UID: "b31e7199-7dd3-4c92-8796-82be9eefb50e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:17:16.673391 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.673346    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld6xg\" (UniqueName: \"kubernetes.io/projected/b31e7199-7dd3-4c92-8796-82be9eefb50e-kube-api-access-ld6xg\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:17:16.673391 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.673382    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:17:16.673391 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:16.673393    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b31e7199-7dd3-4c92-8796-82be9eefb50e-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:17:17.312498 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:17.312472    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"
Oct 31 04:17:17.312675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:17.312476    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p" event={"ID":"b31e7199-7dd3-4c92-8796-82be9eefb50e","Type":"ContainerDied","Data":"eb81e3dbc401be8257cac4d22e0266610fa4e7a593d033163852e26a52b7c427"}
Oct 31 04:17:17.312675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:17.312594    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb81e3dbc401be8257cac4d22e0266610fa4e7a593d033163852e26a52b7c427"
Oct 31 04:17:20.610035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:20.609917    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:17:28.413317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:28.413278    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:17:28.564094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:28.564062    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:17:30.492025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:30.491990    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:17:36.386675 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:17:36.386627    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:17:41.115251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.115195    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"]
Oct 31 04:17:41.115675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.115662    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b31e7199-7dd3-4c92-8796-82be9eefb50e" containerName="revoke-keys"
Oct 31 04:17:41.115722 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.115677    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b31e7199-7dd3-4c92-8796-82be9eefb50e" containerName="revoke-keys"
Oct 31 04:17:41.115761 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.115754    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b31e7199-7dd3-4c92-8796-82be9eefb50e" containerName="revoke-keys"
Oct 31 04:17:41.118888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.118869    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.175175 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.175138    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"]
Oct 31 04:17:41.208160 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.208121    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.208358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.208172    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv7pw\" (UniqueName: \"kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.208358 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.208286    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.223277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.223237    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:17:41.223277 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.223243    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:17:41.309483 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.309442    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.309483 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.309482    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv7pw\" (UniqueName: \"kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.309729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.309503    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.310101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.310081    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.311983 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.311960    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.372987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.372913    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv7pw\" (UniqueName: \"kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw\") pod \"maas-api-revoke-sub-a4aef7d7-drftd\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.431663 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.431622    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:41.805438 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:17:41.805404    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2645a157_f0fd_49f6_b4c8_d8230092951e.slice/crio-41e97a80194e0c7d4b19d1c1946528838df3f484ef961332e17564f5bd6d3c96 WatchSource:0}: Error finding container 41e97a80194e0c7d4b19d1c1946528838df3f484ef961332e17564f5bd6d3c96: Status 404 returned error can't find the container with id 41e97a80194e0c7d4b19d1c1946528838df3f484ef961332e17564f5bd6d3c96
Oct 31 04:17:41.807852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:41.807825    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"]
Oct 31 04:17:42.055987 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:42.055962    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"]
Oct 31 04:17:42.059053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:42.059014    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-52f8d1f7-vchng"]
Oct 31 04:17:42.351422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:42.351342    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6074e71-f4e6-4065-a3bf-6ba7f88d90eb" path="/var/lib/kubelet/pods/e6074e71-f4e6-4065-a3bf-6ba7f88d90eb/volumes"
Oct 31 04:17:42.479862 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:42.479818    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd" event={"ID":"2645a157-f0fd-49f6-b4c8-d8230092951e","Type":"ContainerStarted","Data":"f3dd2aeb139d7fcb0b8928e097868dfa558685bc10888e672c78f57568b2ad06"}
Oct 31 04:17:42.480042 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:42.479885    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd" event={"ID":"2645a157-f0fd-49f6-b4c8-d8230092951e","Type":"ContainerStarted","Data":"41e97a80194e0c7d4b19d1c1946528838df3f484ef961332e17564f5bd6d3c96"}
Oct 31 04:17:42.611455 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:42.611350    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd" podStartSLOduration=1.611331742 podStartE2EDuration="1.611331742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:17:42.60802363 +0000 UTC m=+2912.441883935" watchObservedRunningTime="2026-10-31 04:17:42.611331742 +0000 UTC m=+2912.445192048"
Oct 31 04:17:43.486219 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:43.486183    5906 generic.go:356] "Generic (PLEG): container finished" podID="2645a157-f0fd-49f6-b4c8-d8230092951e" containerID="f3dd2aeb139d7fcb0b8928e097868dfa558685bc10888e672c78f57568b2ad06" exitCode=0
Oct 31 04:17:43.486787 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:43.486269    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd" event={"ID":"2645a157-f0fd-49f6-b4c8-d8230092951e","Type":"ContainerDied","Data":"f3dd2aeb139d7fcb0b8928e097868dfa558685bc10888e672c78f57568b2ad06"}
Oct 31 04:17:44.628709 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.628679    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:44.745724 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.745642    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca\") pod \"2645a157-f0fd-49f6-b4c8-d8230092951e\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") "
Oct 31 04:17:44.745724 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.745682    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca\") pod \"2645a157-f0fd-49f6-b4c8-d8230092951e\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") "
Oct 31 04:17:44.745985 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.745766    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw\" (UniqueName: \"kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw\") pod \"2645a157-f0fd-49f6-b4c8-d8230092951e\" (UID: \"2645a157-f0fd-49f6-b4c8-d8230092951e\") "
Oct 31 04:17:44.746174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.746046    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca" pod "2645a157-f0fd-49f6-b4c8-d8230092951e" (UID: "2645a157-f0fd-49f6-b4c8-d8230092951e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:17:44.746174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.746153    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:17:44.748674 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.748642    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw" pod "2645a157-f0fd-49f6-b4c8-d8230092951e" (UID: "2645a157-f0fd-49f6-b4c8-d8230092951e"). InnerVolumeSpecName "kube-api-access-tv7pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:17:44.749076 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.749047    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca" pod "2645a157-f0fd-49f6-b4c8-d8230092951e" (UID: "2645a157-f0fd-49f6-b4c8-d8230092951e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:17:44.846934 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.846892    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv7pw\" (UniqueName: \"kubernetes.io/projected/2645a157-f0fd-49f6-b4c8-d8230092951e-kube-api-access-tv7pw\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:17:44.846934 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:44.846928    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2645a157-f0fd-49f6-b4c8-d8230092951e-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:17:45.498695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:45.498660    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"
Oct 31 04:17:45.498695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:45.498677    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd" event={"ID":"2645a157-f0fd-49f6-b4c8-d8230092951e","Type":"ContainerDied","Data":"41e97a80194e0c7d4b19d1c1946528838df3f484ef961332e17564f5bd6d3c96"}
Oct 31 04:17:45.498964 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:17:45.498730    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="41e97a80194e0c7d4b19d1c1946528838df3f484ef961332e17564f5bd6d3c96"
Oct 31 04:18:06.381169 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:18:06.381125    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:18:09.618956 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:09.618921    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:18:10.465521 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:10.465483    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:18:11.840860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.840820    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"]
Oct 31 04:18:11.841438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.841319    5906 cpu_manager.go:414] "containerMap: removing container" podUID="2645a157-f0fd-49f6-b4c8-d8230092951e" containerName="revoke-keys"
Oct 31 04:18:11.841438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.841333    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2645a157-f0fd-49f6-b4c8-d8230092951e" containerName="revoke-keys"
Oct 31 04:18:11.841438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.841401    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="2645a157-f0fd-49f6-b4c8-d8230092951e" containerName="revoke-keys"
Oct 31 04:18:11.844637 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.844612    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:11.900066 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.900030    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:18:11.900286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.900171    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:18:11.901819 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.901733    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"]
Oct 31 04:18:11.984173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.984126    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:11.984173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.984173    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q874\" (UniqueName: \"kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:11.984452 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:11.984337    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.085001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.084963    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.085001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.085009    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q874\" (UniqueName: \"kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.085318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.085057    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.085798 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.085772    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.087680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.087658    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.148437 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.148365    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q874\" (UniqueName: \"kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874\") pod \"maas-api-revoke-sub-79805ca9-jpzjv\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.158240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.158194    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:12.530803 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:18:12.530771    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54db60f5_8576_4fd9_98ae_840190f23621.slice/crio-19aaa7888e960f853d163703ccf458abe25763e0d0e7ef42b110935b921390a8 WatchSource:0}: Error finding container 19aaa7888e960f853d163703ccf458abe25763e0d0e7ef42b110935b921390a8: Status 404 returned error can't find the container with id 19aaa7888e960f853d163703ccf458abe25763e0d0e7ef42b110935b921390a8
Oct 31 04:18:12.534719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.534284    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"]
Oct 31 04:18:12.660932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:12.660893    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv" event={"ID":"54db60f5-8576-4fd9-98ae-840190f23621","Type":"ContainerStarted","Data":"19aaa7888e960f853d163703ccf458abe25763e0d0e7ef42b110935b921390a8"}
Oct 31 04:18:13.668601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:13.668564    5906 generic.go:356] "Generic (PLEG): container finished" podID="54db60f5-8576-4fd9-98ae-840190f23621" containerID="5e26e930c4d67b03b4182a1bb5e0352a8a756479dbe4debfb1d9b7a8a6a45be1" exitCode=0
Oct 31 04:18:13.669001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:13.668641    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv" event={"ID":"54db60f5-8576-4fd9-98ae-840190f23621","Type":"ContainerDied","Data":"5e26e930c4d67b03b4182a1bb5e0352a8a756479dbe4debfb1d9b7a8a6a45be1"}
Oct 31 04:18:14.807706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.807678    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:14.915114 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.915074    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca\") pod \"54db60f5-8576-4fd9-98ae-840190f23621\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") "
Oct 31 04:18:14.915346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.915149    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874\" (UniqueName: \"kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874\") pod \"54db60f5-8576-4fd9-98ae-840190f23621\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") "
Oct 31 04:18:14.915346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.915238    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca\") pod \"54db60f5-8576-4fd9-98ae-840190f23621\" (UID: \"54db60f5-8576-4fd9-98ae-840190f23621\") "
Oct 31 04:18:14.915496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.915471    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca" pod "54db60f5-8576-4fd9-98ae-840190f23621" (UID: "54db60f5-8576-4fd9-98ae-840190f23621"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:18:14.915676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.915661    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:14.917410 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.917387    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca" pod "54db60f5-8576-4fd9-98ae-840190f23621" (UID: "54db60f5-8576-4fd9-98ae-840190f23621"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:18:14.917488 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:14.917470    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874" pod "54db60f5-8576-4fd9-98ae-840190f23621" (UID: "54db60f5-8576-4fd9-98ae-840190f23621"). InnerVolumeSpecName "kube-api-access-9q874". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:18:15.016487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:15.016446    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q874\" (UniqueName: \"kubernetes.io/projected/54db60f5-8576-4fd9-98ae-840190f23621-kube-api-access-9q874\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:15.016487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:15.016482    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54db60f5-8576-4fd9-98ae-840190f23621-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:15.680665 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:15.680634    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"
Oct 31 04:18:15.680853 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:15.680667    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv" event={"ID":"54db60f5-8576-4fd9-98ae-840190f23621","Type":"ContainerDied","Data":"19aaa7888e960f853d163703ccf458abe25763e0d0e7ef42b110935b921390a8"}
Oct 31 04:18:15.680853 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:15.680718    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="19aaa7888e960f853d163703ccf458abe25763e0d0e7ef42b110935b921390a8"
Oct 31 04:18:21.310549 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:21.310499    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:18:22.156696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:22.156658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:18:30.867567 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:30.867529    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:18:35.052223 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:35.052176    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"]
Oct 31 04:18:35.091087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:35.091050    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85237e45-jmqsq"]
Oct 31 04:18:36.356555 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:36.356514    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aad4b4df-0d22-49e0-ace2-436a21523927" path="/var/lib/kubelet/pods/aad4b4df-0d22-49e0-ace2-436a21523927/volumes"
Oct 31 04:18:36.381232 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:18:36.381173    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:18:42.559774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.559735    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"]
Oct 31 04:18:42.560227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.560187    5906 cpu_manager.go:414] "containerMap: removing container" podUID="54db60f5-8576-4fd9-98ae-840190f23621" containerName="revoke-keys"
Oct 31 04:18:42.560285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.560232    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54db60f5-8576-4fd9-98ae-840190f23621" containerName="revoke-keys"
Oct 31 04:18:42.560367 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.560356    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="54db60f5-8576-4fd9-98ae-840190f23621" containerName="revoke-keys"
Oct 31 04:18:42.563490 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.563472    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.618371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.618336    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"]
Oct 31 04:18:42.643636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.643596    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.643810 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.643758    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6wrf\" (UniqueName: \"kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.643920 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.643903    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.667849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.667814    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:18:42.668013 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.667903    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:18:42.744544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.744509    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6wrf\" (UniqueName: \"kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.744736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.744610    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.744736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.744646    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.745362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.745336    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.747031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.746995    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.807826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.807787    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6wrf\" (UniqueName: \"kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf\") pod \"maas-api-revoke-sub-0b7e8edb-8hhtf\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:42.876821 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:42.876738    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:43.281558 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:18:43.281521    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29765d88_e9fd_4a80_9251_3f7d726133b2.slice/crio-a422d30c7536778256269ee68ca99bd7e784a07bd3cfbf024e714b24cf6744e0 WatchSource:0}: Error finding container a422d30c7536778256269ee68ca99bd7e784a07bd3cfbf024e714b24cf6744e0: Status 404 returned error can't find the container with id a422d30c7536778256269ee68ca99bd7e784a07bd3cfbf024e714b24cf6744e0
Oct 31 04:18:43.281696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:43.281631    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"]
Oct 31 04:18:43.837929 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:43.837840    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf" event={"ID":"29765d88-e9fd-4a80-9251-3f7d726133b2","Type":"ContainerStarted","Data":"8a15ecda270407c62367791a1ba57d9e4cd0b65140b64eaca5c641080e9b0477"}
Oct 31 04:18:43.837929 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:43.837891    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf" event={"ID":"29765d88-e9fd-4a80-9251-3f7d726133b2","Type":"ContainerStarted","Data":"a422d30c7536778256269ee68ca99bd7e784a07bd3cfbf024e714b24cf6744e0"}
Oct 31 04:18:43.957714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:43.957657    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf" podStartSLOduration=1.957637766 podStartE2EDuration="1.957637766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:18:43.956513788 +0000 UTC m=+2973.790374093" watchObservedRunningTime="2026-10-31 04:18:43.957637766 +0000 UTC m=+2973.791498071"
Oct 31 04:18:44.844412 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:44.844372    5906 generic.go:356] "Generic (PLEG): container finished" podID="29765d88-e9fd-4a80-9251-3f7d726133b2" containerID="8a15ecda270407c62367791a1ba57d9e4cd0b65140b64eaca5c641080e9b0477" exitCode=0
Oct 31 04:18:44.844832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:44.844469    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf" event={"ID":"29765d88-e9fd-4a80-9251-3f7d726133b2","Type":"ContainerDied","Data":"8a15ecda270407c62367791a1ba57d9e4cd0b65140b64eaca5c641080e9b0477"}
Oct 31 04:18:45.998647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:45.998615    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:46.077563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.077525    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf\" (UniqueName: \"kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf\") pod \"29765d88-e9fd-4a80-9251-3f7d726133b2\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") "
Oct 31 04:18:46.077563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.077571    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca\") pod \"29765d88-e9fd-4a80-9251-3f7d726133b2\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") "
Oct 31 04:18:46.077796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.077705    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca\") pod \"29765d88-e9fd-4a80-9251-3f7d726133b2\" (UID: \"29765d88-e9fd-4a80-9251-3f7d726133b2\") "
Oct 31 04:18:46.078014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.077985    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca" pod "29765d88-e9fd-4a80-9251-3f7d726133b2" (UID: "29765d88-e9fd-4a80-9251-3f7d726133b2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:18:46.079757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.079732    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf" pod "29765d88-e9fd-4a80-9251-3f7d726133b2" (UID: "29765d88-e9fd-4a80-9251-3f7d726133b2"). InnerVolumeSpecName "kube-api-access-k6wrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:18:46.079876 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.079780    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca" pod "29765d88-e9fd-4a80-9251-3f7d726133b2" (UID: "29765d88-e9fd-4a80-9251-3f7d726133b2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:18:46.179230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.179119    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:46.179230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.179153    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6wrf\" (UniqueName: \"kubernetes.io/projected/29765d88-e9fd-4a80-9251-3f7d726133b2-kube-api-access-k6wrf\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:46.179230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.179163    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/29765d88-e9fd-4a80-9251-3f7d726133b2-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:46.856025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.855993    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"
Oct 31 04:18:46.856239 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.856033    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf" event={"ID":"29765d88-e9fd-4a80-9251-3f7d726133b2","Type":"ContainerDied","Data":"a422d30c7536778256269ee68ca99bd7e784a07bd3cfbf024e714b24cf6744e0"}
Oct 31 04:18:46.856239 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:46.856091    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a422d30c7536778256269ee68ca99bd7e784a07bd3cfbf024e714b24cf6744e0"
Oct 31 04:18:53.999993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:53.999907    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"]
Oct 31 04:18:54.000577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.000556    5906 cpu_manager.go:414] "containerMap: removing container" podUID="29765d88-e9fd-4a80-9251-3f7d726133b2" containerName="revoke-keys"
Oct 31 04:18:54.000624 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.000578    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29765d88-e9fd-4a80-9251-3f7d726133b2" containerName="revoke-keys"
Oct 31 04:18:54.000665 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.000656    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="29765d88-e9fd-4a80-9251-3f7d726133b2" containerName="revoke-keys"
Oct 31 04:18:54.003945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.003928    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.060863 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.060831    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.061071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.060902    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.061071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.060953    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zskwx\" (UniqueName: \"kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.064439 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.064405    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"]
Oct 31 04:18:54.112365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.112306    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:18:54.112365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.112361    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:18:54.162072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.162034    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.162303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.162113    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.162303 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.162148    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zskwx\" (UniqueName: \"kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.162840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.162816    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.164590 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.164570    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.225556 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.225526    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zskwx\" (UniqueName: \"kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx\") pod \"maas-api-revoke-sub-3d4156f1-bf8kk\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.316932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.316844    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:54.688683 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:18:54.688643    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2938fb76_67a1_4c83_af08_45e80831e6f2.slice/crio-3f31a6ae42ce8dd9097eebe38df6e2f24c3c6bc828f9c331cf74917fb626d158 WatchSource:0}: Error finding container 3f31a6ae42ce8dd9097eebe38df6e2f24c3c6bc828f9c331cf74917fb626d158: Status 404 returned error can't find the container with id 3f31a6ae42ce8dd9097eebe38df6e2f24c3c6bc828f9c331cf74917fb626d158
Oct 31 04:18:54.690630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.690602    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"]
Oct 31 04:18:54.898271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:54.898227    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk" event={"ID":"2938fb76-67a1-4c83-af08-45e80831e6f2","Type":"ContainerStarted","Data":"3f31a6ae42ce8dd9097eebe38df6e2f24c3c6bc828f9c331cf74917fb626d158"}
Oct 31 04:18:55.929371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:55.929315    5906 generic.go:356] "Generic (PLEG): container finished" podID="2938fb76-67a1-4c83-af08-45e80831e6f2" containerID="35bbca3c9190b4e6c6e4235489b3cdfbedc79e00e36e019f6b7e26d04726278a" exitCode=0
Oct 31 04:18:55.929870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:55.929495    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk" event={"ID":"2938fb76-67a1-4c83-af08-45e80831e6f2","Type":"ContainerDied","Data":"35bbca3c9190b4e6c6e4235489b3cdfbedc79e00e36e019f6b7e26d04726278a"}
Oct 31 04:18:57.714689 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.714380    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:57.768953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.768893    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca\") pod \"2938fb76-67a1-4c83-af08-45e80831e6f2\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") "
Oct 31 04:18:57.769185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.769018    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx\" (UniqueName: \"kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx\") pod \"2938fb76-67a1-4c83-af08-45e80831e6f2\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") "
Oct 31 04:18:57.769185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.769100    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca\") pod \"2938fb76-67a1-4c83-af08-45e80831e6f2\" (UID: \"2938fb76-67a1-4c83-af08-45e80831e6f2\") "
Oct 31 04:18:57.769864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.769834    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca" pod "2938fb76-67a1-4c83-af08-45e80831e6f2" (UID: "2938fb76-67a1-4c83-af08-45e80831e6f2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:18:57.772584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.772539    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca" pod "2938fb76-67a1-4c83-af08-45e80831e6f2" (UID: "2938fb76-67a1-4c83-af08-45e80831e6f2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:18:57.774232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.774161    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx" pod "2938fb76-67a1-4c83-af08-45e80831e6f2" (UID: "2938fb76-67a1-4c83-af08-45e80831e6f2"). InnerVolumeSpecName "kube-api-access-zskwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:18:57.870744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.870651    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zskwx\" (UniqueName: \"kubernetes.io/projected/2938fb76-67a1-4c83-af08-45e80831e6f2-kube-api-access-zskwx\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:57.870744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.870687    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:57.870744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.870697    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2938fb76-67a1-4c83-af08-45e80831e6f2-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:18:57.943710 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.943678    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"
Oct 31 04:18:57.943710 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.943692    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk" event={"ID":"2938fb76-67a1-4c83-af08-45e80831e6f2","Type":"ContainerDied","Data":"3f31a6ae42ce8dd9097eebe38df6e2f24c3c6bc828f9c331cf74917fb626d158"}
Oct 31 04:18:57.943928 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:18:57.943744    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f31a6ae42ce8dd9097eebe38df6e2f24c3c6bc828f9c331cf74917fb626d158"
Oct 31 04:19:06.383534 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:19:06.383494    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:19:11.642918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:11.642883    5906 scope.go:122] "RemoveContainer" containerID="d9f86ab6fc877fec89e825535ad80a45cc006c8044d309a7a99480311589aa5e"
Oct 31 04:19:11.652062 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:11.652035    5906 scope.go:122] "RemoveContainer" containerID="39ab75533347127351c0f1e50e78ccb11d3db703f1d0edb482b6c8e9e5f7b438"
Oct 31 04:19:11.917179 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:11.917086    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"]
Oct 31 04:19:11.968817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:11.968775    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-bf8kk"]
Oct 31 04:19:12.032791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.032759    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"]
Oct 31 04:19:12.033213 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.033190    5906 cpu_manager.go:414] "containerMap: removing container" podUID="2938fb76-67a1-4c83-af08-45e80831e6f2" containerName="revoke-keys"
Oct 31 04:19:12.033265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.033225    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2938fb76-67a1-4c83-af08-45e80831e6f2" containerName="revoke-keys"
Oct 31 04:19:12.033337 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.033327    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="2938fb76-67a1-4c83-af08-45e80831e6f2" containerName="revoke-keys"
Oct 31 04:19:12.038101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.038082    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.092410 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.092372    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"]
Oct 31 04:19:12.092610 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.092441    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:19:12.092610 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.092524    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:19:12.123170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.123137    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn2ff\" (UniqueName: \"kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.123401 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.123377    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.123450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.123433    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.224658 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.224569    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn2ff\" (UniqueName: \"kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.224833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.224683    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.224833 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.224716    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.225490 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.225462    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.227140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.227116    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.288189 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.288155    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn2ff\" (UniqueName: \"kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff\") pod \"maas-api-revoke-sub-3d4156f1-42bmh\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.350695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.350663    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:12.350866 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.350826    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2938fb76-67a1-4c83-af08-45e80831e6f2" path="/var/lib/kubelet/pods/2938fb76-67a1-4c83-af08-45e80831e6f2/volumes"
Oct 31 04:19:12.725261 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:19:12.725222    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93bdb8a4_ddd6_4c68_b8d9_f85bf348a49b.slice/crio-5a072dfa41dc17ec15697d63395ee1cadceb3eb4a39feaca3c55df5aa80e0ac7 WatchSource:0}: Error finding container 5a072dfa41dc17ec15697d63395ee1cadceb3eb4a39feaca3c55df5aa80e0ac7: Status 404 returned error can't find the container with id 5a072dfa41dc17ec15697d63395ee1cadceb3eb4a39feaca3c55df5aa80e0ac7
Oct 31 04:19:12.727565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:12.727537    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"]
Oct 31 04:19:13.028888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:13.028854    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh" event={"ID":"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b","Type":"ContainerStarted","Data":"5a072dfa41dc17ec15697d63395ee1cadceb3eb4a39feaca3c55df5aa80e0ac7"}
Oct 31 04:19:14.035423 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:14.035388    5906 generic.go:356] "Generic (PLEG): container finished" podID="93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" containerID="e523de5f2ad38ebcdedc63c6314bd751f14e4ed5d976412639884c6f24b933f2" exitCode=0
Oct 31 04:19:14.035835 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:14.035471    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh" event={"ID":"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b","Type":"ContainerDied","Data":"e523de5f2ad38ebcdedc63c6314bd751f14e4ed5d976412639884c6f24b933f2"}
Oct 31 04:19:15.190366 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.190336    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:15.253630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.253596    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff\" (UniqueName: \"kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff\") pod \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") "
Oct 31 04:19:15.253802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.253677    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca\") pod \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") "
Oct 31 04:19:15.253802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.253732    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca\") pod \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\" (UID: \"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b\") "
Oct 31 04:19:15.254108 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.254074    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca" pod "93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" (UID: "93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:19:15.255763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.255739    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca" pod "93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" (UID: "93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:19:15.255852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.255744    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff" pod "93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" (UID: "93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b"). InnerVolumeSpecName "kube-api-access-rn2ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:19:15.354612 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.354529    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:19:15.354612 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.354560    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:19:15.354612 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:15.354570    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn2ff\" (UniqueName: \"kubernetes.io/projected/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b-kube-api-access-rn2ff\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:19:16.047811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:16.047777    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh" event={"ID":"93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b","Type":"ContainerDied","Data":"5a072dfa41dc17ec15697d63395ee1cadceb3eb4a39feaca3c55df5aa80e0ac7"}
Oct 31 04:19:16.047811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:16.047813    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"
Oct 31 04:19:16.048040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:16.047824    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5a072dfa41dc17ec15697d63395ee1cadceb3eb4a39feaca3c55df5aa80e0ac7"
Oct 31 04:19:31.215241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:19:31.215166    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:19:36.393017 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:19:36.392984    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:20:06.484186 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:20:06.484127    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:20:11.678498 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:11.678459    5906 scope.go:122] "RemoveContainer" containerID="f3db8e3a39dcda372f62b8ad0b8267b0244606fe42f03b6ff5d810445d5b72fb"
Oct 31 04:20:13.585779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.585740    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"]
Oct 31 04:20:13.586235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.586201    5906 cpu_manager.go:414] "containerMap: removing container" podUID="93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" containerName="revoke-keys"
Oct 31 04:20:13.586235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.586229    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" containerName="revoke-keys"
Oct 31 04:20:13.586326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.586317    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" containerName="revoke-keys"
Oct 31 04:20:13.590899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.590883    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.619091 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.619055    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.619251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.619101    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc754\" (UniqueName: \"kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.619251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.619201    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.645093 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.645057    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"]
Oct 31 04:20:13.693984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.693950    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:20:13.694164 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.694033    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:20:13.720312 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.720271    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.720312 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.720322    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc754\" (UniqueName: \"kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.720515 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.720383    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.721055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.721029    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.722610 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.722589    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.783334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.783300    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc754\" (UniqueName: \"kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754\") pod \"maas-api-revoke-sub-e2fc9c5b-lwqqd\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:13.904281 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:13.904176    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:14.274120 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:20:14.274086    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6de07c_250c_4e36_965e_0e4030bd959a.slice/crio-cbcd8d06a9b0033633813ceda4a40d2d325a76d4935ebf39ee0e813bd0ac26eb WatchSource:0}: Error finding container cbcd8d06a9b0033633813ceda4a40d2d325a76d4935ebf39ee0e813bd0ac26eb: Status 404 returned error can't find the container with id cbcd8d06a9b0033633813ceda4a40d2d325a76d4935ebf39ee0e813bd0ac26eb
Oct 31 04:20:14.275978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:14.275953    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"]
Oct 31 04:20:14.636511 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:14.636467    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd" event={"ID":"1a6de07c-250c-4e36-965e-0e4030bd959a","Type":"ContainerStarted","Data":"84574e1ca95feee6a38a664e7204ef33cbd99aa674f1ac8a1c295ef91da2e925"}
Oct 31 04:20:14.636895 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:14.636535    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd" event={"ID":"1a6de07c-250c-4e36-965e-0e4030bd959a","Type":"ContainerStarted","Data":"cbcd8d06a9b0033633813ceda4a40d2d325a76d4935ebf39ee0e813bd0ac26eb"}
Oct 31 04:20:14.756911 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:14.756851    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd" podStartSLOduration=1.7568381990000002 podStartE2EDuration="1.756838199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:20:14.754479815 +0000 UTC m=+3064.588340119" watchObservedRunningTime="2026-10-31 04:20:14.756838199 +0000 UTC m=+3064.590698639"
Oct 31 04:20:15.643066 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:15.643028    5906 generic.go:356] "Generic (PLEG): container finished" podID="1a6de07c-250c-4e36-965e-0e4030bd959a" containerID="84574e1ca95feee6a38a664e7204ef33cbd99aa674f1ac8a1c295ef91da2e925" exitCode=0
Oct 31 04:20:15.643510 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:15.643112    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd" event={"ID":"1a6de07c-250c-4e36-965e-0e4030bd959a","Type":"ContainerDied","Data":"84574e1ca95feee6a38a664e7204ef33cbd99aa674f1ac8a1c295ef91da2e925"}
Oct 31 04:20:16.788876 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.788848    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:16.851331 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.851269    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754\" (UniqueName: \"kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754\") pod \"1a6de07c-250c-4e36-965e-0e4030bd959a\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") "
Oct 31 04:20:16.851527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.851341    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca\") pod \"1a6de07c-250c-4e36-965e-0e4030bd959a\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") "
Oct 31 04:20:16.851527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.851394    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca\") pod \"1a6de07c-250c-4e36-965e-0e4030bd959a\" (UID: \"1a6de07c-250c-4e36-965e-0e4030bd959a\") "
Oct 31 04:20:16.851691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.851667    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca" pod "1a6de07c-250c-4e36-965e-0e4030bd959a" (UID: "1a6de07c-250c-4e36-965e-0e4030bd959a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:20:16.851837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.851755    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:16.853572 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.853536    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca" pod "1a6de07c-250c-4e36-965e-0e4030bd959a" (UID: "1a6de07c-250c-4e36-965e-0e4030bd959a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:20:16.853691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.853614    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754" pod "1a6de07c-250c-4e36-965e-0e4030bd959a" (UID: "1a6de07c-250c-4e36-965e-0e4030bd959a"). InnerVolumeSpecName "kube-api-access-qc754". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:20:16.952718 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.952623    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qc754\" (UniqueName: \"kubernetes.io/projected/1a6de07c-250c-4e36-965e-0e4030bd959a-kube-api-access-qc754\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:16.952718 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:16.952660    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1a6de07c-250c-4e36-965e-0e4030bd959a-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:17.655324 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:17.655292    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"
Oct 31 04:20:17.655537 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:17.655290    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd" event={"ID":"1a6de07c-250c-4e36-965e-0e4030bd959a","Type":"ContainerDied","Data":"cbcd8d06a9b0033633813ceda4a40d2d325a76d4935ebf39ee0e813bd0ac26eb"}
Oct 31 04:20:17.655537 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:17.655415    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cbcd8d06a9b0033633813ceda4a40d2d325a76d4935ebf39ee0e813bd0ac26eb"
Oct 31 04:20:24.295700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.295622    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"]
Oct 31 04:20:24.296131 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.296116    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1a6de07c-250c-4e36-965e-0e4030bd959a" containerName="revoke-keys"
Oct 31 04:20:24.296185 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.296132    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a6de07c-250c-4e36-965e-0e4030bd959a" containerName="revoke-keys"
Oct 31 04:20:24.296245 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.296226    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a6de07c-250c-4e36-965e-0e4030bd959a" containerName="revoke-keys"
Oct 31 04:20:24.299724 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.299699    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.357379 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.357346    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"]
Oct 31 04:20:24.405992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.405954    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:20:24.406187 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.406083    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:20:24.430005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.429964    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qspcf\" (UniqueName: \"kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.430230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.430060    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.430329 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.430302    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.531995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.531952    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qspcf\" (UniqueName: \"kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.532198 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.532012    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.532198 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.532034    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.532822 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.532796    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.534583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.534558    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.595238 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.595142    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qspcf\" (UniqueName: \"kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf\") pod \"maas-api-revoke-sub-5a29efb3-hpsws\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.614019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.613984    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:24.985594 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:20:24.985556    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54289b29_356f_4fa7_af99_c9b9c9353930.slice/crio-17eb66fece97b32d623346e29f76370acb3f362e475778d30ec148283c6bd024 WatchSource:0}: Error finding container 17eb66fece97b32d623346e29f76370acb3f362e475778d30ec148283c6bd024: Status 404 returned error can't find the container with id 17eb66fece97b32d623346e29f76370acb3f362e475778d30ec148283c6bd024
Oct 31 04:20:24.988342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:24.988314    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"]
Oct 31 04:20:25.700563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:25.700470    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws" event={"ID":"54289b29-356f-4fa7-af99-c9b9c9353930","Type":"ContainerStarted","Data":"40eb1534a1314cd9c8cc163c0495249e12144afa692e494ccbfa926dca5ab55b"}
Oct 31 04:20:25.700563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:25.700529    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws" event={"ID":"54289b29-356f-4fa7-af99-c9b9c9353930","Type":"ContainerStarted","Data":"17eb66fece97b32d623346e29f76370acb3f362e475778d30ec148283c6bd024"}
Oct 31 04:20:25.820670 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:25.820612    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws" podStartSLOduration=1.8205900879999999 podStartE2EDuration="1.820590088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:20:25.820090811 +0000 UTC m=+3075.653951116" watchObservedRunningTime="2026-10-31 04:20:25.820590088 +0000 UTC m=+3075.654450396"
Oct 31 04:20:26.709618 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:26.709577    5906 generic.go:356] "Generic (PLEG): container finished" podID="54289b29-356f-4fa7-af99-c9b9c9353930" containerID="40eb1534a1314cd9c8cc163c0495249e12144afa692e494ccbfa926dca5ab55b" exitCode=0
Oct 31 04:20:26.710096 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:26.709721    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws" event={"ID":"54289b29-356f-4fa7-af99-c9b9c9353930","Type":"ContainerDied","Data":"40eb1534a1314cd9c8cc163c0495249e12144afa692e494ccbfa926dca5ab55b"}
Oct 31 04:20:28.439844 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.439423    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:28.441843 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.441808    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca\") pod \"54289b29-356f-4fa7-af99-c9b9c9353930\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") "
Oct 31 04:20:28.442117 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.442093    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca\") pod \"54289b29-356f-4fa7-af99-c9b9c9353930\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") "
Oct 31 04:20:28.450465 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.450435    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf\" (UniqueName: \"kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf\") pod \"54289b29-356f-4fa7-af99-c9b9c9353930\" (UID: \"54289b29-356f-4fa7-af99-c9b9c9353930\") "
Oct 31 04:20:28.472276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.443936    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca" pod "54289b29-356f-4fa7-af99-c9b9c9353930" (UID: "54289b29-356f-4fa7-af99-c9b9c9353930"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:20:28.472533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.454828    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca" pod "54289b29-356f-4fa7-af99-c9b9c9353930" (UID: "54289b29-356f-4fa7-af99-c9b9c9353930"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:20:28.499438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.499374    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf" pod "54289b29-356f-4fa7-af99-c9b9c9353930" (UID: "54289b29-356f-4fa7-af99-c9b9c9353930"). InnerVolumeSpecName "kube-api-access-qspcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:20:28.579226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.579012    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:28.579226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.579060    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/54289b29-356f-4fa7-af99-c9b9c9353930-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:28.579226 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.579078    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qspcf\" (UniqueName: \"kubernetes.io/projected/54289b29-356f-4fa7-af99-c9b9c9353930-kube-api-access-qspcf\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:28.851968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.851928    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws" event={"ID":"54289b29-356f-4fa7-af99-c9b9c9353930","Type":"ContainerDied","Data":"17eb66fece97b32d623346e29f76370acb3f362e475778d30ec148283c6bd024"}
Oct 31 04:20:28.886745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.886614    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17eb66fece97b32d623346e29f76370acb3f362e475778d30ec148283c6bd024"
Oct 31 04:20:28.892369 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:28.875382    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"
Oct 31 04:20:31.516650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:31.516614    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:20:36.382126 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:20:36.382091    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:20:41.148130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.148092    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"]
Oct 31 04:20:41.199784 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.199754    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-42bmh"]
Oct 31 04:20:41.262308 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.262270    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"]
Oct 31 04:20:41.262729 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.262715    5906 cpu_manager.go:414] "containerMap: removing container" podUID="54289b29-356f-4fa7-af99-c9b9c9353930" containerName="revoke-keys"
Oct 31 04:20:41.262780 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.262730    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54289b29-356f-4fa7-af99-c9b9c9353930" containerName="revoke-keys"
Oct 31 04:20:41.262852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.262843    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="54289b29-356f-4fa7-af99-c9b9c9353930" containerName="revoke-keys"
Oct 31 04:20:41.266227 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.266191    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.320932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.320887    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:20:41.320932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.320903    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:20:41.321873 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.321851    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"]
Oct 31 04:20:41.451809 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.451716    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.451984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.451875    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjfr\" (UniqueName: \"kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.451984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.451938    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.553038 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.553000    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.553242 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.553116    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjfr\" (UniqueName: \"kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.553242 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.553150    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.553819 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.553793    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.555450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.555428    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.616077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.616043    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjfr\" (UniqueName: \"kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr\") pod \"maas-api-revoke-sub-3d4156f1-mnv6c\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:41.879691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:41.879657    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:42.250136 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:20:42.250100    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6854ac2a_c701_456e_af9e_352df1892528.slice/crio-40237f0c593f53ff78d4d7fa91ac2c3c8237165f7294db8e7aeb1f101e906fe2 WatchSource:0}: Error finding container 40237f0c593f53ff78d4d7fa91ac2c3c8237165f7294db8e7aeb1f101e906fe2: Status 404 returned error can't find the container with id 40237f0c593f53ff78d4d7fa91ac2c3c8237165f7294db8e7aeb1f101e906fe2
Oct 31 04:20:42.252760 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:42.252732    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"]
Oct 31 04:20:42.351347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:42.351310    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b" path="/var/lib/kubelet/pods/93bdb8a4-ddd6-4c68-b8d9-f85bf348a49b/volumes"
Oct 31 04:20:42.932199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:42.932107    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c" event={"ID":"6854ac2a-c701-456e-af9e-352df1892528","Type":"ContainerStarted","Data":"005ba37a61cdcfe0a3b581054e90e79f9f94baa30f1734e29a526cf9f4447b58"}
Oct 31 04:20:42.932199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:42.932165    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c" event={"ID":"6854ac2a-c701-456e-af9e-352df1892528","Type":"ContainerStarted","Data":"40237f0c593f53ff78d4d7fa91ac2c3c8237165f7294db8e7aeb1f101e906fe2"}
Oct 31 04:20:43.050991 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:43.050951    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"]
Oct 31 04:20:43.102368 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:43.102318    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c" podStartSLOduration=2.102304229 podStartE2EDuration="2.102304229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:20:43.1005682 +0000 UTC m=+3092.934428507" watchObservedRunningTime="2026-10-31 04:20:43.102304229 +0000 UTC m=+3092.936164533"
Oct 31 04:20:43.104860 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:43.104834    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2e1f5b5f-78xr2"]
Oct 31 04:20:43.939940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:43.939899    5906 generic.go:356] "Generic (PLEG): container finished" podID="6854ac2a-c701-456e-af9e-352df1892528" containerID="005ba37a61cdcfe0a3b581054e90e79f9f94baa30f1734e29a526cf9f4447b58" exitCode=0
Oct 31 04:20:43.940435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:43.939964    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c" event={"ID":"6854ac2a-c701-456e-af9e-352df1892528","Type":"ContainerDied","Data":"005ba37a61cdcfe0a3b581054e90e79f9f94baa30f1734e29a526cf9f4447b58"}
Oct 31 04:20:44.351986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:44.351948    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6d7dfe2-e358-4fc9-ae0d-c61c4337b319" path="/var/lib/kubelet/pods/e6d7dfe2-e358-4fc9-ae0d-c61c4337b319/volumes"
Oct 31 04:20:45.082318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.082294    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:45.193544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.193508    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr\" (UniqueName: \"kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr\") pod \"6854ac2a-c701-456e-af9e-352df1892528\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") "
Oct 31 04:20:45.193736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.193555    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca\") pod \"6854ac2a-c701-456e-af9e-352df1892528\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") "
Oct 31 04:20:45.193736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.193694    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca\") pod \"6854ac2a-c701-456e-af9e-352df1892528\" (UID: \"6854ac2a-c701-456e-af9e-352df1892528\") "
Oct 31 04:20:45.194001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.193974    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca" pod "6854ac2a-c701-456e-af9e-352df1892528" (UID: "6854ac2a-c701-456e-af9e-352df1892528"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:20:45.194101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.194081    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6854ac2a-c701-456e-af9e-352df1892528-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:45.195829 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.195765    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca" pod "6854ac2a-c701-456e-af9e-352df1892528" (UID: "6854ac2a-c701-456e-af9e-352df1892528"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:20:45.195829 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.195798    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr" pod "6854ac2a-c701-456e-af9e-352df1892528" (UID: "6854ac2a-c701-456e-af9e-352df1892528"). InnerVolumeSpecName "kube-api-access-pfjfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:20:45.294804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.294773    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfjfr\" (UniqueName: \"kubernetes.io/projected/6854ac2a-c701-456e-af9e-352df1892528-kube-api-access-pfjfr\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:45.294804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.294801    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6854ac2a-c701-456e-af9e-352df1892528-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:20:45.952286 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.952252    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"
Oct 31 04:20:45.952499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.952251    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c" event={"ID":"6854ac2a-c701-456e-af9e-352df1892528","Type":"ContainerDied","Data":"40237f0c593f53ff78d4d7fa91ac2c3c8237165f7294db8e7aeb1f101e906fe2"}
Oct 31 04:20:45.952499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:45.952382    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40237f0c593f53ff78d4d7fa91ac2c3c8237165f7294db8e7aeb1f101e906fe2"
Oct 31 04:20:56.051002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:56.050966    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"]
Oct 31 04:20:56.102432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:56.102395    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ec843018-8pmgp"]
Oct 31 04:20:56.351501 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:20:56.351409    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30751e83-bd2c-4163-b126-324e96587302" path="/var/lib/kubelet/pods/30751e83-bd2c-4163-b126-324e96587302/volumes"
Oct 31 04:21:04.126536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.126490    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:04.683590 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.683546    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"]
Oct 31 04:21:04.684169 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.684147    5906 cpu_manager.go:414] "containerMap: removing container" podUID="6854ac2a-c701-456e-af9e-352df1892528" containerName="revoke-keys"
Oct 31 04:21:04.684169 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.684168    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6854ac2a-c701-456e-af9e-352df1892528" containerName="revoke-keys"
Oct 31 04:21:04.684405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.684296    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="6854ac2a-c701-456e-af9e-352df1892528" containerName="revoke-keys"
Oct 31 04:21:04.703661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.703625    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.751301 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.751266    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:04.758027 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.757993    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-f2eca2-openshift-default-dockercfg-cm946\""
Oct 31 04:21:04.804236 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.804167    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.804643 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.804610    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.804907 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.804877    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.805022 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.804949    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.805022 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.805005    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.805165 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.805091    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmhbj\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.805271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.805163    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.805271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.805187    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.805271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.805227    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.818154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.818112    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"]
Oct 31 04:21:04.906700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.906659    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.906917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.906722    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmhbj\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.906917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.906783    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.906917 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.906807    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.906942    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907016    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907081    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907172    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907258    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907310 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907269    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907479 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907309    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907520    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907557    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.907622 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.907587    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.909279 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.909258    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.909584 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.909566    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.975686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.975614    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:04.976772 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:04.976742    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmhbj\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj\") pod \"e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") " pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:05.005240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:05.005192    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:05.025113 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:05.025079    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:05.449842 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:21:05.449749    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8dbe321_4257_474f_a178_5cca44a94bc9.slice/crio-dc7ef2053715d8e91a484e282650392b8b109778c546e2f3fb702f9e88737511 WatchSource:0}: Error finding container dc7ef2053715d8e91a484e282650392b8b109778c546e2f3fb702f9e88737511: Status 404 returned error can't find the container with id dc7ef2053715d8e91a484e282650392b8b109778c546e2f3fb702f9e88737511
Oct 31 04:21:05.451252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:05.451227    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"]
Oct 31 04:21:05.452082 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:05.452054    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:21:05.452135 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:05.452118    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:21:05.452228 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:05.452150    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:21:06.060779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:06.060737    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" event={"ID":"c8dbe321-4257-474f-a178-5cca44a94bc9","Type":"ContainerStarted","Data":"dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"}
Oct 31 04:21:06.060970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:06.060793    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" event={"ID":"c8dbe321-4257-474f-a178-5cca44a94bc9","Type":"ContainerStarted","Data":"dc7ef2053715d8e91a484e282650392b8b109778c546e2f3fb702f9e88737511"}
Oct 31 04:21:06.186414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:06.186353    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podStartSLOduration=2.186335905 podStartE2EDuration="2.186335905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:21:06.184587382 +0000 UTC m=+3116.018447687" watchObservedRunningTime="2026-10-31 04:21:06.186335905 +0000 UTC m=+3116.020196215"
Oct 31 04:21:06.381498 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:21:06.381411    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:21:07.026306 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:07.026269    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:07.031723 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:07.031689    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:07.065699 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:07.065666    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:07.066615 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:07.066590    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:19.045032 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:19.044995    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:19.212931 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:19.212889    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:19.356373 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:19.356285    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:22.169125 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:22.169087    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:22.329965 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:22.329931    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:24.874456 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.874416    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"]
Oct 31 04:21:24.879491 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.879466    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:24.936766 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.936722    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"]
Oct 31 04:21:24.941249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.941216    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94cpn\" (UniqueName: \"kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:24.941398 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.941307    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:24.941398 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.941388    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:24.941480 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.941431    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:24.986002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.985968    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gc447\""
Oct 31 04:21:24.986002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.985987    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-f2eca2-serving-cert\""
Oct 31 04:21:24.986276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:24.986032    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 04:21:25.042052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.042017    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.042265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.042131    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94cpn\" (UniqueName: \"kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.042265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.042161    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.042265 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.042234    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.042540 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:21:25.042310    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-serving-cert: secret "maas-api-e2e-worker-single-main-f2eca2-serving-cert" not found
Oct 31 04:21:25.042540 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:21:25.042391    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls podName:be8c4a1c-48c0-475c-a4f5-4f35a9d64554 nodeName:}" failed. No retries permitted until 2026-10-31 04:21:25.542368339 +0000 UTC m=+3135.376228642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls") pod "maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" (UID: "be8c4a1c-48c0-475c-a4f5-4f35a9d64554") : secret "maas-api-e2e-worker-single-main-f2eca2-serving-cert" not found
Oct 31 04:21:25.042884 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.042865    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.044565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.044545    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.052051 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.052021    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:25.105661 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.105630    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94cpn\" (UniqueName: \"kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.248592 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.248561    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:25.387624 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.387591    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:25.548529 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.548431    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.551017 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.550984    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:25.792136 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:25.792093    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:26.167139 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:21:26.167103    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe8c4a1c_48c0_475c_a4f5_4f35a9d64554.slice/crio-69e814cf88bd4b2bc07cc78c4acf8725993d479002263fa1c974a9e0efd0247f WatchSource:0}: Error finding container 69e814cf88bd4b2bc07cc78c4acf8725993d479002263fa1c974a9e0efd0247f: Status 404 returned error can't find the container with id 69e814cf88bd4b2bc07cc78c4acf8725993d479002263fa1c974a9e0efd0247f
Oct 31 04:21:26.168871 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:26.168846    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:21:26.170049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:26.170024    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"]
Oct 31 04:21:27.177342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:27.177307    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" event={"ID":"be8c4a1c-48c0-475c-a4f5-4f35a9d64554","Type":"ContainerStarted","Data":"69e814cf88bd4b2bc07cc78c4acf8725993d479002263fa1c974a9e0efd0247f"}
Oct 31 04:21:28.184054 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:28.184015    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" event={"ID":"be8c4a1c-48c0-475c-a4f5-4f35a9d64554","Type":"ContainerStarted","Data":"89a7071f646db386faf73b115a25e8ea1577696c30110c2125f557a216094be1"}
Oct 31 04:21:28.184500 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:28.184101    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:28.305302 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:28.305250    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" podStartSLOduration=3.055382913 podStartE2EDuration="4.305234041s" totalImagesPullingTime="1.249851128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:21:28.30487936 +0000 UTC m=+3138.138739665" watchObservedRunningTime="2026-10-31 04:21:28.305234041 +0000 UTC m=+3138.139094354"
Oct 31 04:21:29.716217 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:29.716169    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:31.845040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:31.845002    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:32.048853 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:32.048815    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"]
Oct 31 04:21:32.100508 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:32.100429    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-046e1343-rkdbd"]
Oct 31 04:21:32.351017 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:32.350924    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4661cb4-28ca-4060-8caa-419725abe3f3" path="/var/lib/kubelet/pods/a4661cb4-28ca-4060-8caa-419725abe3f3/volumes"
Oct 31 04:21:34.036785 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:34.036750    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:34.173507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:34.173473    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:34.194771 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:34.194743    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:21:34.319467 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:34.319387    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:36.393496 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:21:36.393456    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:21:38.815437 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:38.815402    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:42.051216 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:42.051179    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"]
Oct 31 04:21:42.102831 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:42.102800    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d3fba056-m955j"]
Oct 31 04:21:42.352002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:42.351927    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f87c85cf-848d-4d9a-a19b-f6db38fc7448" path="/var/lib/kubelet/pods/f87c85cf-848d-4d9a-a19b-f6db38fc7448/volumes"
Oct 31 04:21:43.286904 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.286869    5906 generic.go:356] "Generic (PLEG): container finished" podID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerID="dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77" exitCode=0
Oct 31 04:21:43.287429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.286953    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" event={"ID":"c8dbe321-4257-474f-a178-5cca44a94bc9","Type":"ContainerDied","Data":"dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"}
Oct 31 04:21:43.287429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.287396    5906 scope.go:122] "RemoveContainer" containerID="dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"
Oct 31 04:21:43.288588 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.288560    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:21:43.288665 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.288633    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:21:43.288703 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.288667    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:21:43.597292 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.597168    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:43.962299 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:43.960637    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:44.294783 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:44.294752    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" event={"ID":"c8dbe321-4257-474f-a178-5cca44a94bc9","Type":"ContainerStarted","Data":"267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86"}
Oct 31 04:21:44.567131 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:44.567041    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:44.708776 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:44.708744    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:45.025496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:45.025459    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:45.025740 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:45.025713    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 04:21:45.025832 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:45.025772    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 04:21:46.026456 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:46.026420    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 04:21:46.026856 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:46.026485    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 04:21:47.025978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:47.025936    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 04:21:47.026173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:47.026015    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 04:21:48.026246 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:48.026192    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 04:21:48.026734 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:48.026279    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 04:21:49.025626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:49.025540    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 31 04:21:49.025626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:49.025610    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 31 04:21:49.335314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:49.335183    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:50.030195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:50.030160    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:50.030651 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:50.030632    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:50.031290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:50.031267    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:21:52.787003 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:52.786968    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:54.711389 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:54.711357    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:57.170087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:57.170050    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:21:57.312977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:21:57.312945    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:01.891702 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:01.891649    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:02.224507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:02.224419    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:06.382407 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:22:06.382370    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:22:06.400105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:06.400071    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:11.007975 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:11.007934    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:11.710691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:11.710656    5906 scope.go:122] "RemoveContainer" containerID="c63eef5d3b7b0a584345c0b06fa8bdcbc0d35994c4157a1be08846176bb68992"
Oct 31 04:22:11.719754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:11.719731    5906 scope.go:122] "RemoveContainer" containerID="32a9800574c324c4a51aa1195ae8df10b2afeae3cc3be39265fec542e88bfab5"
Oct 31 04:22:15.503746 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:15.503708    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:17.060909 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:17.060877    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"]
Oct 31 04:22:17.112010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:17.111976    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ae594979-7n59p"]
Oct 31 04:22:18.352230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:18.352175    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b31e7199-7dd3-4c92-8796-82be9eefb50e" path="/var/lib/kubelet/pods/b31e7199-7dd3-4c92-8796-82be9eefb50e/volumes"
Oct 31 04:22:20.010139 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:20.010101    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:24.498736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:24.498699    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:29.001475 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:29.001428    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:32.188055 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:32.188022    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:32.327709 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:32.327673    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:33.510873 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:33.510836    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:36.381004 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:22:36.380960    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:22:38.098894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:38.098857    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:42.596287 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:42.596192    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:45.059294 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:45.059256    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"]
Oct 31 04:22:45.110522 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:45.110488    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-drftd"]
Oct 31 04:22:46.350874 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:46.350838    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2645a157-f0fd-49f6-b4c8-d8230092951e" path="/var/lib/kubelet/pods/2645a157-f0fd-49f6-b4c8-d8230092951e/volumes"
Oct 31 04:22:47.250556 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:47.250504    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:51.706793 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:51.706759    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:55.138438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:55.138402    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:22:56.497345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:22:56.497302    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:00.295643 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:00.295610    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:01.434663 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:01.434629    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:05.983074 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:05.983040    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:06.380779 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:23:06.380740    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:23:07.355178 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:07.355137    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:07.496634 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:07.496603    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:10.487780 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:10.487745    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:11.749132 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:11.749097    5906 scope.go:122] "RemoveContainer" containerID="9915249e65fdb1a09400067e42574169ffea560f8d71cbf81e46a766d93493a8"
Oct 31 04:23:11.757920 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:11.757900    5906 scope.go:122] "RemoveContainer" containerID="ccdc0cd633834ca93e6536e12ff0416a05a3c05c639ce82309a252583b90a55b"
Oct 31 04:23:14.437630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:14.437594    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"]
Oct 31 04:23:14.438138 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:14.437833    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" podUID="be8c4a1c-48c0-475c-a4f5-4f35a9d64554" containerName="maas-api" containerID="cri-o://89a7071f646db386faf73b115a25e8ea1577696c30110c2125f557a216094be1" gracePeriod=30
Oct 31 04:23:14.564115 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:14.564079    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:14.812796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:14.812760    5906 generic.go:356] "Generic (PLEG): container finished" podID="be8c4a1c-48c0-475c-a4f5-4f35a9d64554" containerID="89a7071f646db386faf73b115a25e8ea1577696c30110c2125f557a216094be1" exitCode=0
Oct 31 04:23:14.812981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:14.812836    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" event={"ID":"be8c4a1c-48c0-475c-a4f5-4f35a9d64554","Type":"ContainerDied","Data":"89a7071f646db386faf73b115a25e8ea1577696c30110c2125f557a216094be1"}
Oct 31 04:23:14.912823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:14.912796    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:23:15.006675 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.006644    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\") pod \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") "
Oct 31 04:23:15.006889 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.006733    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs\") pod \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") "
Oct 31 04:23:15.006889 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.006828    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca\") pod \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") "
Oct 31 04:23:15.007040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.006914    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn\" (UniqueName: \"kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn\") pod \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\" (UID: \"be8c4a1c-48c0-475c-a4f5-4f35a9d64554\") "
Oct 31 04:23:15.007211 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.007180    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca" pod "be8c4a1c-48c0-475c-a4f5-4f35a9d64554" (UID: "be8c4a1c-48c0-475c-a4f5-4f35a9d64554"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:23:15.007330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.007311    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.008912 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.008891    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs" pod "be8c4a1c-48c0-475c-a4f5-4f35a9d64554" (UID: "be8c4a1c-48c0-475c-a4f5-4f35a9d64554"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:23:15.008996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.008945    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn" pod "be8c4a1c-48c0-475c-a4f5-4f35a9d64554" (UID: "be8c4a1c-48c0-475c-a4f5-4f35a9d64554"). InnerVolumeSpecName "kube-api-access-94cpn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:23:15.008996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.008952    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls" pod "be8c4a1c-48c0-475c-a4f5-4f35a9d64554" (UID: "be8c4a1c-48c0-475c-a4f5-4f35a9d64554"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:23:15.091026 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.090992    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"]
Oct 31 04:23:15.092632 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.092599    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-jpzjv"]
Oct 31 04:23:15.108945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.108916    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.108945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.108947    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.109162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.108963    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94cpn\" (UniqueName: \"kubernetes.io/projected/be8c4a1c-48c0-475c-a4f5-4f35a9d64554-kube-api-access-94cpn\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.819668 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.819642    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"
Oct 31 04:23:15.820173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.819678    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9" event={"ID":"be8c4a1c-48c0-475c-a4f5-4f35a9d64554","Type":"ContainerDied","Data":"69e814cf88bd4b2bc07cc78c4acf8725993d479002263fa1c974a9e0efd0247f"}
Oct 31 04:23:15.820173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.819744    5906 scope.go:122] "RemoveContainer" containerID="89a7071f646db386faf73b115a25e8ea1577696c30110c2125f557a216094be1"
Oct 31 04:23:15.980579 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.980543    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:15.990648 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:15.990617    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"]
Oct 31 04:23:16.041586 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:16.041555    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-f2eca2-57776dfb7c-ltwc9"]
Oct 31 04:23:16.245213 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:16.245184    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:16.352609 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:16.352576    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54db60f5-8576-4fd9-98ae-840190f23621" path="/var/lib/kubelet/pods/54db60f5-8576-4fd9-98ae-840190f23621/volumes"
Oct 31 04:23:16.353026 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:16.353010    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be8c4a1c-48c0-475c-a4f5-4f35a9d64554" path="/var/lib/kubelet/pods/be8c4a1c-48c0-475c-a4f5-4f35a9d64554/volumes"
Oct 31 04:23:17.022146 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:17.022119    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:23.236840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:23.236765    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:23.377981 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:23.377947    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:23.587135 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:23.587098    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:23.594746 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:23.594714    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"]
Oct 31 04:23:23.594961 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:23.594934    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy" containerID="cri-o://267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86" gracePeriod=30
Oct 31 04:23:25.164162 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.164128    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"]
Oct 31 04:23:25.215506 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.215468    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-mnv6c"]
Oct 31 04:23:25.279423 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.279391    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"]
Oct 31 04:23:25.279924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.279904    5906 cpu_manager.go:414] "containerMap: removing container" podUID="be8c4a1c-48c0-475c-a4f5-4f35a9d64554" containerName="maas-api"
Oct 31 04:23:25.280024 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.279927    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be8c4a1c-48c0-475c-a4f5-4f35a9d64554" containerName="maas-api"
Oct 31 04:23:25.280081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.280061    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="be8c4a1c-48c0-475c-a4f5-4f35a9d64554" containerName="maas-api"
Oct 31 04:23:25.284639 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.284617    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.323372 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.323333    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.323536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.323464    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.323536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.323504    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm4sk\" (UniqueName: \"kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.341558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.341525    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:23:25.342357 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.342335    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:23:25.348851 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.348807    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"]
Oct 31 04:23:25.424447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.424352    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.424447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.424405    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm4sk\" (UniqueName: \"kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.424700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.424529    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.425116 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.425076    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.426870 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.426851    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.487352 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.487324    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm4sk\" (UniqueName: \"kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk\") pod \"maas-api-revoke-sub-3d4156f1-g6kt2\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.598500 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.598464    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:25.971808 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:23:25.971770    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ca5118_5788_44f1_a5c4_9beea2a6a49a.slice/crio-7f018224dd06c3520d604b8bc45402112c703b5a58f9b8b92a924d93e98ad252 WatchSource:0}: Error finding container 7f018224dd06c3520d604b8bc45402112c703b5a58f9b8b92a924d93e98ad252: Status 404 returned error can't find the container with id 7f018224dd06c3520d604b8bc45402112c703b5a58f9b8b92a924d93e98ad252
Oct 31 04:23:25.973201 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:25.973171    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"]
Oct 31 04:23:26.351596 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:26.351554    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6854ac2a-c701-456e-af9e-352df1892528" path="/var/lib/kubelet/pods/6854ac2a-c701-456e-af9e-352df1892528/volumes"
Oct 31 04:23:26.882189 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:26.882157    5906 generic.go:356] "Generic (PLEG): container finished" podID="b7ca5118-5788-44f1-a5c4-9beea2a6a49a" containerID="7d6a568c3a31fe1d231941c9b167362952124eb6ce11ff390200b45c18501863" exitCode=0
Oct 31 04:23:26.882378 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:26.882222    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2" event={"ID":"b7ca5118-5788-44f1-a5c4-9beea2a6a49a","Type":"ContainerDied","Data":"7d6a568c3a31fe1d231941c9b167362952124eb6ce11ff390200b45c18501863"}
Oct 31 04:23:26.882378 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:26.882275    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2" event={"ID":"b7ca5118-5788-44f1-a5c4-9beea2a6a49a","Type":"ContainerStarted","Data":"7f018224dd06c3520d604b8bc45402112c703b5a58f9b8b92a924d93e98ad252"}
Oct 31 04:23:28.028528 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.028502    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:28.154839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.154758    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca\") pod \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") "
Oct 31 04:23:28.155015 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.154877    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk\" (UniqueName: \"kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk\") pod \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") "
Oct 31 04:23:28.155015 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.154996    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca\") pod \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\" (UID: \"b7ca5118-5788-44f1-a5c4-9beea2a6a49a\") "
Oct 31 04:23:28.155273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.155249    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca" pod "b7ca5118-5788-44f1-a5c4-9beea2a6a49a" (UID: "b7ca5118-5788-44f1-a5c4-9beea2a6a49a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:23:28.155600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.155565    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.157330 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.157307    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca" pod "b7ca5118-5788-44f1-a5c4-9beea2a6a49a" (UID: "b7ca5118-5788-44f1-a5c4-9beea2a6a49a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:23:28.157668 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.157639    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk" pod "b7ca5118-5788-44f1-a5c4-9beea2a6a49a" (UID: "b7ca5118-5788-44f1-a5c4-9beea2a6a49a"). InnerVolumeSpecName "kube-api-access-hm4sk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:23:28.257191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.257150    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm4sk\" (UniqueName: \"kubernetes.io/projected/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-kube-api-access-hm4sk\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.257191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.257185    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7ca5118-5788-44f1-a5c4-9beea2a6a49a-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.828813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.828785    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:23:28.894679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.894592    5906 generic.go:356] "Generic (PLEG): container finished" podID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerID="267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86" exitCode=0
Oct 31 04:23:28.894679 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.894639    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"
Oct 31 04:23:28.894885 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.894678    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" event={"ID":"c8dbe321-4257-474f-a178-5cca44a94bc9","Type":"ContainerDied","Data":"267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86"}
Oct 31 04:23:28.894885 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.894742    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb" event={"ID":"c8dbe321-4257-474f-a178-5cca44a94bc9","Type":"ContainerDied","Data":"dc7ef2053715d8e91a484e282650392b8b109778c546e2f3fb702f9e88737511"}
Oct 31 04:23:28.894885 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.894777    5906 scope.go:122] "RemoveContainer" containerID="267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86"
Oct 31 04:23:28.896140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.895942    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2" event={"ID":"b7ca5118-5788-44f1-a5c4-9beea2a6a49a","Type":"ContainerDied","Data":"7f018224dd06c3520d604b8bc45402112c703b5a58f9b8b92a924d93e98ad252"}
Oct 31 04:23:28.896140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.895977    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f018224dd06c3520d604b8bc45402112c703b5a58f9b8b92a924d93e98ad252"
Oct 31 04:23:28.896140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.895983    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"
Oct 31 04:23:28.904333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.904313    5906 scope.go:122] "RemoveContainer" containerID="dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"
Oct 31 04:23:28.913079 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.913057    5906 scope.go:122] "RemoveContainer" containerID="267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86"
Oct 31 04:23:28.913425 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:23:28.913404    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86\": container with ID starting with 267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86 not found: ID does not exist" containerID="267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86"
Oct 31 04:23:28.913501 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.913437    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86"} err="failed to get container status \"267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86\": rpc error: code = NotFound desc = could not find container \"267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86\": container with ID starting with 267f92c7297dd58648e7f62b934cb69a45486907d88105128d9610da3d516b86 not found: ID does not exist"
Oct 31 04:23:28.913501 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.913462    5906 scope.go:122] "RemoveContainer" containerID="dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"
Oct 31 04:23:28.913747 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:23:28.913730    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77\": container with ID starting with dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77 not found: ID does not exist" containerID="dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"
Oct 31 04:23:28.913791 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.913754    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77"} err="failed to get container status \"dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77\": rpc error: code = NotFound desc = could not find container \"dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77\": container with ID starting with dc5099313c90d300ceaebac8f1ddca7dd4c68f6723d23c9f2648001abe38ca77 not found: ID does not exist"
Oct 31 04:23:28.963249 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963186    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963424 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963280    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963424 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963364    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963424 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963392    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963428    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963483    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963589 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963548    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963594    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963636    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj\") pod \"c8dbe321-4257-474f-a178-5cca44a94bc9\" (UID: \"c8dbe321-4257-474f-a178-5cca44a94bc9\") "
Oct 31 04:23:28.963759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963696    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:28.963759 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963715    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:28.963995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963804    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:28.963995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963821    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:23:28.963995 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.963942    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:28.964179 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.964161    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.964273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.964184    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.964273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.964232    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8dbe321-4257-474f-a178-5cca44a94bc9-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.964273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.964248    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.964273 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.964262    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:28.965828 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.965808    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "kube-api-access-mmhbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:23:28.965879 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.965830    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:28.965969 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.965945    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:23:28.965969 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:28.965958    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token" pod "c8dbe321-4257-474f-a178-5cca44a94bc9" (UID: "c8dbe321-4257-474f-a178-5cca44a94bc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:23:29.065454 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:29.065420    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:29.065454 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:29.065446    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:29.065454 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:29.065456    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8dbe321-4257-474f-a178-5cca44a94bc9-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:29.065906 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:29.065467    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmhbj\" (UniqueName: \"kubernetes.io/projected/c8dbe321-4257-474f-a178-5cca44a94bc9-kube-api-access-mmhbj\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:29.320918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:29.320880    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"]
Oct 31 04:23:29.325918 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:29.325887    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-f2eca2-openshift-default-895c54957-pkfxb"]
Oct 31 04:23:30.357992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:30.357942    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" path="/var/lib/kubelet/pods/c8dbe321-4257-474f-a178-5cca44a94bc9/volumes"
Oct 31 04:23:32.546035 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:32.546001    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:36.382243 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:23:36.382196    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:23:37.223786 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:37.223752    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:38.095221 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:38.095184    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:44.445808 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:44.445769    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:23:46.059599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:46.059558    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"]
Oct 31 04:23:46.111023 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:46.110979    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-8hhtf"]
Oct 31 04:23:46.350637 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:23:46.350560    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29765d88-e9fd-4a80-9251-3f7d726133b2" path="/var/lib/kubelet/pods/29765d88-e9fd-4a80-9251-3f7d726133b2/volumes"
Oct 31 04:24:06.381719 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:24:06.381674    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:24:11.782370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:24:11.782338    5906 scope.go:122] "RemoveContainer" containerID="d3ce5f15e754df0b883f27e085987f20158210483d3121267a5e87386c247a01"
Oct 31 04:24:11.791288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:24:11.791270    5906 scope.go:122] "RemoveContainer" containerID="f3dd2aeb139d7fcb0b8928e097868dfa558685bc10888e672c78f57568b2ad06"
Oct 31 04:24:32.946068 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:24:32.945988    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:24:36.381941 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:24:36.381901    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:25:06.381374 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:25:06.381285    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:25:11.823447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:11.823409    5906 scope.go:122] "RemoveContainer" containerID="35bbca3c9190b4e6c6e4235489b3cdfbedc79e00e36e019f6b7e26d04726278a"
Oct 31 04:25:11.832219 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:11.832179    5906 scope.go:122] "RemoveContainer" containerID="8a15ecda270407c62367791a1ba57d9e4cd0b65140b64eaca5c641080e9b0477"
Oct 31 04:25:11.841805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:11.841777    5906 scope.go:122] "RemoveContainer" containerID="5e26e930c4d67b03b4182a1bb5e0352a8a756479dbe4debfb1d9b7a8a6a45be1"
Oct 31 04:25:17.060244 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:17.060191    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"]
Oct 31 04:25:17.111487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:17.111450    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e2fc9c5b-lwqqd"]
Oct 31 04:25:18.354544 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:18.354500    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a6de07c-250c-4e36-965e-0e4030bd959a" path="/var/lib/kubelet/pods/1a6de07c-250c-4e36-965e-0e4030bd959a/volumes"
Oct 31 04:25:21.686533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686496    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"]
Oct 31 04:25:21.686947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686919    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy"
Oct 31 04:25:21.686947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686930    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy"
Oct 31 04:25:21.686947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686940    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy"
Oct 31 04:25:21.686947 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686945    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy"
Oct 31 04:25:21.687097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686968    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b7ca5118-5788-44f1-a5c4-9beea2a6a49a" containerName="revoke-keys"
Oct 31 04:25:21.687097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.686982    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7ca5118-5788-44f1-a5c4-9beea2a6a49a" containerName="revoke-keys"
Oct 31 04:25:21.687097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.687050    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy"
Oct 31 04:25:21.687097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.687061    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7ca5118-5788-44f1-a5c4-9beea2a6a49a" containerName="revoke-keys"
Oct 31 04:25:21.691915 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.691898    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.744301 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.744264    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"]
Oct 31 04:25:21.794312 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.794274    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:25:21.794507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.794319    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:25:21.848201 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.848161    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.848427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.848301    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckvqz\" (UniqueName: \"kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.848427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.848391    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.949854 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.949751    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckvqz\" (UniqueName: \"kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.949854 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.949804    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.950123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.949903    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.950632 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.950607    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:21.952370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:21.952338    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:22.012578 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:22.012536    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckvqz\" (UniqueName: \"kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz\") pod \"maas-api-revoke-sub-d576e68b-lhl7k\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:22.305583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:22.305545    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:22.674416 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:25:22.674383    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b7e959_2762_496e_8826_fa4460254b35.slice/crio-35e95ac10ba740471e31fc8c2bc7ae1daecfdf1753f36f88d839d559f6251d43 WatchSource:0}: Error finding container 35e95ac10ba740471e31fc8c2bc7ae1daecfdf1753f36f88d839d559f6251d43: Status 404 returned error can't find the container with id 35e95ac10ba740471e31fc8c2bc7ae1daecfdf1753f36f88d839d559f6251d43
Oct 31 04:25:22.675726 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:22.675697    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"]
Oct 31 04:25:23.494400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:23.494365    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k" event={"ID":"45b7e959-2762-496e-8826-fa4460254b35","Type":"ContainerStarted","Data":"ad13a2d85c2ebffca1125a057c0c4af036ef4dda9855a5c3baf14aafe5da1086"}
Oct 31 04:25:23.494839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:23.494421    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k" event={"ID":"45b7e959-2762-496e-8826-fa4460254b35","Type":"ContainerStarted","Data":"35e95ac10ba740471e31fc8c2bc7ae1daecfdf1753f36f88d839d559f6251d43"}
Oct 31 04:25:23.613295 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:23.613238    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k" podStartSLOduration=2.613191554 podStartE2EDuration="2.613191554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:25:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:25:23.612960749 +0000 UTC m=+3373.446821055" watchObservedRunningTime="2026-10-31 04:25:23.613191554 +0000 UTC m=+3373.447051859"
Oct 31 04:25:24.501695 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:24.501658    5906 generic.go:356] "Generic (PLEG): container finished" podID="45b7e959-2762-496e-8826-fa4460254b35" containerID="ad13a2d85c2ebffca1125a057c0c4af036ef4dda9855a5c3baf14aafe5da1086" exitCode=0
Oct 31 04:25:24.502093 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:24.501745    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k" event={"ID":"45b7e959-2762-496e-8826-fa4460254b35","Type":"ContainerDied","Data":"ad13a2d85c2ebffca1125a057c0c4af036ef4dda9855a5c3baf14aafe5da1086"}
Oct 31 04:25:25.641778 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.641751    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:25.790801 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.790709    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz\" (UniqueName: \"kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz\") pod \"45b7e959-2762-496e-8826-fa4460254b35\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") "
Oct 31 04:25:25.790996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.790812    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca\") pod \"45b7e959-2762-496e-8826-fa4460254b35\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") "
Oct 31 04:25:25.790996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.790866    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca\") pod \"45b7e959-2762-496e-8826-fa4460254b35\" (UID: \"45b7e959-2762-496e-8826-fa4460254b35\") "
Oct 31 04:25:25.791235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.791173    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca" pod "45b7e959-2762-496e-8826-fa4460254b35" (UID: "45b7e959-2762-496e-8826-fa4460254b35"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:25:25.791347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.791322    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/45b7e959-2762-496e-8826-fa4460254b35-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:25:25.792922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.792899    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz" pod "45b7e959-2762-496e-8826-fa4460254b35" (UID: "45b7e959-2762-496e-8826-fa4460254b35"). InnerVolumeSpecName "kube-api-access-ckvqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:25:25.792922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.792911    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca" pod "45b7e959-2762-496e-8826-fa4460254b35" (UID: "45b7e959-2762-496e-8826-fa4460254b35"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:25:25.892757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.892718    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/45b7e959-2762-496e-8826-fa4460254b35-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:25:25.892757 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:25.892751    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckvqz\" (UniqueName: \"kubernetes.io/projected/45b7e959-2762-496e-8826-fa4460254b35-kube-api-access-ckvqz\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:25:26.512563 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:26.512533    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"
Oct 31 04:25:26.512744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:26.512534    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k" event={"ID":"45b7e959-2762-496e-8826-fa4460254b35","Type":"ContainerDied","Data":"35e95ac10ba740471e31fc8c2bc7ae1daecfdf1753f36f88d839d559f6251d43"}
Oct 31 04:25:26.512744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:26.512634    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35e95ac10ba740471e31fc8c2bc7ae1daecfdf1753f36f88d839d559f6251d43"
Oct 31 04:25:28.050323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:28.050287    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"]
Oct 31 04:25:28.101559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:28.101522    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-5a29efb3-hpsws"]
Oct 31 04:25:28.350924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:28.350843    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54289b29-356f-4fa7-af99-c9b9c9353930" path="/var/lib/kubelet/pods/54289b29-356f-4fa7-af99-c9b9c9353930/volumes"
Oct 31 04:25:33.224305 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:33.224269    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:25:33.360628 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:33.360595    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:25:36.380530 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:25:36.380494    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:25:40.884837 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:40.884790    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:25:50.680040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.680006    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"]
Oct 31 04:25:50.680524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.680464    5906 cpu_manager.go:414] "containerMap: removing container" podUID="45b7e959-2762-496e-8826-fa4460254b35" containerName="revoke-keys"
Oct 31 04:25:50.680524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.680478    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45b7e959-2762-496e-8826-fa4460254b35" containerName="revoke-keys"
Oct 31 04:25:50.680606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.680549    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="45b7e959-2762-496e-8826-fa4460254b35" containerName="revoke-keys"
Oct 31 04:25:50.680606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.680559    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8dbe321-4257-474f-a178-5cca44a94bc9" containerName="istio-proxy"
Oct 31 04:25:50.683733 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.683715    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.738560 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.738526    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"]
Oct 31 04:25:50.788300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.788261    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:25:50.788497 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.788325    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:25:50.838047 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.838008    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.838262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.838129    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8lfw\" (UniqueName: \"kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.838262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.838170    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.939576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.939481    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.939576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.939570    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8lfw\" (UniqueName: \"kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.939796 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.939597    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.940370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.940343    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:50.942264 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:50.942236    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:51.003597 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:51.003554    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8lfw\" (UniqueName: \"kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw\") pod \"maas-api-revoke-sub-d2bc93bc-l2gwb\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:51.296400 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:51.296364    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:51.673797 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:51.673768    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"]
Oct 31 04:25:52.682476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:52.682439    5906 generic.go:356] "Generic (PLEG): container finished" podID="b3e57001-e010-4cba-94e3-6eabe4da8e9e" containerID="f0d577f40014d5ccd42917db354ce2ddf66491f6a8bfbd786193006177af67fb" exitCode=0
Oct 31 04:25:52.682925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:52.682513    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb" event={"ID":"b3e57001-e010-4cba-94e3-6eabe4da8e9e","Type":"ContainerDied","Data":"f0d577f40014d5ccd42917db354ce2ddf66491f6a8bfbd786193006177af67fb"}
Oct 31 04:25:52.682925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:52.682567    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb" event={"ID":"b3e57001-e010-4cba-94e3-6eabe4da8e9e","Type":"ContainerStarted","Data":"233752292fd3678aad24054fe2b23daa0c4fbafa33a15f165fd90d9f77a4c229"}
Oct 31 04:25:53.833007 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.832981    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:53.869949 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.869916    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca\") pod \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") "
Oct 31 04:25:53.870123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.870021    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca\") pod \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") "
Oct 31 04:25:53.870123 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.870075    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw\" (UniqueName: \"kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw\") pod \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\" (UID: \"b3e57001-e010-4cba-94e3-6eabe4da8e9e\") "
Oct 31 04:25:53.870336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.870312    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca" pod "b3e57001-e010-4cba-94e3-6eabe4da8e9e" (UID: "b3e57001-e010-4cba-94e3-6eabe4da8e9e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:25:53.870504 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.870483    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:25:53.872199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.872173    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca" pod "b3e57001-e010-4cba-94e3-6eabe4da8e9e" (UID: "b3e57001-e010-4cba-94e3-6eabe4da8e9e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:25:53.872362 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.872338    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw" pod "b3e57001-e010-4cba-94e3-6eabe4da8e9e" (UID: "b3e57001-e010-4cba-94e3-6eabe4da8e9e"). InnerVolumeSpecName "kube-api-access-p8lfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:25:53.971425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.971334    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b3e57001-e010-4cba-94e3-6eabe4da8e9e-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:25:53.971425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:53.971365    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8lfw\" (UniqueName: \"kubernetes.io/projected/b3e57001-e010-4cba-94e3-6eabe4da8e9e-kube-api-access-p8lfw\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:25:54.693970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:54.693885    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"
Oct 31 04:25:54.693970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:54.693910    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb" event={"ID":"b3e57001-e010-4cba-94e3-6eabe4da8e9e","Type":"ContainerDied","Data":"233752292fd3678aad24054fe2b23daa0c4fbafa33a15f165fd90d9f77a4c229"}
Oct 31 04:25:54.693970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:25:54.693958    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="233752292fd3678aad24054fe2b23daa0c4fbafa33a15f165fd90d9f77a4c229"
Oct 31 04:26:06.382138 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:26:06.382096    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:26:11.872002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:11.871970    5906 scope.go:122] "RemoveContainer" containerID="e523de5f2ad38ebcdedc63c6314bd751f14e4ed5d976412639884c6f24b933f2"
Oct 31 04:26:13.074279 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.074242    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:13.900133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.900096    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"]
Oct 31 04:26:13.900570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.900557    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b3e57001-e010-4cba-94e3-6eabe4da8e9e" containerName="revoke-keys"
Oct 31 04:26:13.900635 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.900572    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3e57001-e010-4cba-94e3-6eabe4da8e9e" containerName="revoke-keys"
Oct 31 04:26:13.900681 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.900637    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3e57001-e010-4cba-94e3-6eabe4da8e9e" containerName="revoke-keys"
Oct 31 04:26:13.909692 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.909656    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.964470 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.964440    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-b097c5-openshift-default-dockercfg-dp5fm\""
Oct 31 04:26:13.982877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.982843    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983060 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.982885    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983060 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.982989    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9d4n\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983060 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.983030    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.983102    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.983131    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.983165    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.983270    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.983316 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.983309    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:13.990921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:13.990879    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:14.017952 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.017907    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"]
Oct 31 04:26:14.084219 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084168    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084247    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084299    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9d4n\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084328    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084367    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084394    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084432    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084578    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.084669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084651    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.085159 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084710    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.085159 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084733    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.085159 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.084848    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.085159 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.085140    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.085417 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.085376    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.086655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.086636    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.086971 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.086954    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.147414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.147380    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.147580 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.147477    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9d4n\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n\") pod \"e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") " pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.222425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.222338    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:14.247051 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.247015    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:14.669364 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:26:14.669333    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93a6a3d_fd40_4238_b9ea_ea4cf75626cd.slice/crio-8258638002d9e301509ca8fd2b3a6a9012bc29a92242cd9c40b91d461d5a03bb WatchSource:0}: Error finding container 8258638002d9e301509ca8fd2b3a6a9012bc29a92242cd9c40b91d461d5a03bb: Status 404 returned error can't find the container with id 8258638002d9e301509ca8fd2b3a6a9012bc29a92242cd9c40b91d461d5a03bb
Oct 31 04:26:14.671804 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.671775    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"]
Oct 31 04:26:14.671989 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.671961    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:26:14.672076 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.672036    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:26:14.672143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.672079    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:26:14.806823 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.806788    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" event={"ID":"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd","Type":"ContainerStarted","Data":"6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"}
Oct 31 04:26:14.807021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.806845    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" event={"ID":"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd","Type":"ContainerStarted","Data":"8258638002d9e301509ca8fd2b3a6a9012bc29a92242cd9c40b91d461d5a03bb"}
Oct 31 04:26:14.931510 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:14.931413    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podStartSLOduration=1.931397194 podStartE2EDuration="1.931397194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:26:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:26:14.930711241 +0000 UTC m=+3424.764571548" watchObservedRunningTime="2026-10-31 04:26:14.931397194 +0000 UTC m=+3424.765257557"
Oct 31 04:26:15.223499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:15.223409    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:15.228419 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:15.228395    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:15.812276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:15.812245    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:15.813163 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:15.813142    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:17.760548 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:17.760508    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:17.917892 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:17.917855    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:18.065106 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:18.065022    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:20.829824 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:20.829740    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:20.992840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:20.992774    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:23.496859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.496827    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"]
Oct 31 04:26:23.500826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.500804    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.606640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.606606    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-b097c5-serving-cert\""
Oct 31 04:26:23.607179 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.607151    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 04:26:23.607323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.607180    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gc447\""
Oct 31 04:26:23.609191 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.609160    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"]
Oct 31 04:26:23.663755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.663706    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:23.700020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.699980    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.700195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.700035    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.700195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.700141    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.700195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.700177    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdvc6\" (UniqueName: \"kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.801393 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.801294    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.801393 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.801342    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.801635 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.801461    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.801635 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.801485    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdvc6\" (UniqueName: \"kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.801635 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:26:23.801465    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-serving-cert: secret "maas-api-e2e-worker-single-main-b097c5-serving-cert" not found
Oct 31 04:26:23.801796 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:26:23.801633    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls podName:9cb6c7be-cf47-4c24-b5d4-88d683adc7b4 nodeName:}" failed. No retries permitted until 2026-10-31 04:26:24.301607953 +0000 UTC m=+3434.135468251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls") pod "maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" (UID: "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4") : secret "maas-api-e2e-worker-single-main-b097c5-serving-cert" not found
Oct 31 04:26:23.802108 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.802090    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.803953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.803934    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:23.817327 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.817295    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:23.873650 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:23.873552    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdvc6\" (UniqueName: \"kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:24.072418 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:24.072331    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:24.307305 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:24.307270    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:24.309824 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:24.309796    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:24.415243 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:24.415138    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:24.790617 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:26:24.790579    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb6c7be_cf47_4c24_b5d4_88d683adc7b4.slice/crio-809187a9cf7276e175b455f6543a22f727016bce422467704ac0f5b755b8a17a WatchSource:0}: Error finding container 809187a9cf7276e175b455f6543a22f727016bce422467704ac0f5b755b8a17a: Status 404 returned error can't find the container with id 809187a9cf7276e175b455f6543a22f727016bce422467704ac0f5b755b8a17a
Oct 31 04:26:24.791496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:24.791470    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"]
Oct 31 04:26:24.874994 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:24.874952    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" event={"ID":"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4","Type":"ContainerStarted","Data":"809187a9cf7276e175b455f6543a22f727016bce422467704ac0f5b755b8a17a"}
Oct 31 04:26:26.885805 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:26.885768    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" event={"ID":"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4","Type":"ContainerStarted","Data":"a23853e2e5eaf249daa3ef4c6a5b45f13d68136416218f33449d29a51024ce85"}
Oct 31 04:26:26.886256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:26.885849    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:27.045727 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:27.045668    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" podStartSLOduration=2.944489533 podStartE2EDuration="4.045652708s" totalImagesPullingTime="1.101163175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:26:27.044263146 +0000 UTC m=+3436.878123451" watchObservedRunningTime="2026-10-31 04:26:27.045652708 +0000 UTC m=+3436.879513021"
Oct 31 04:26:28.313022 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:28.312985    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:32.892588 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:32.892552    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:32.896503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:32.896478    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:26:33.558231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:33.556927    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:36.395552 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:26:36.395516    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:26:37.415138 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:37.415053    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:40.955775 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:40.955744    5906 generic.go:356] "Generic (PLEG): container finished" podID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerID="6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71" exitCode=0
Oct 31 04:26:40.956269 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:40.955814    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" event={"ID":"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd","Type":"ContainerDied","Data":"6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"}
Oct 31 04:26:40.956269 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:40.956232    5906 scope.go:122] "RemoveContainer" containerID="6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"
Oct 31 04:26:40.957655 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:40.957631    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:26:40.957742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:40.957696    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:26:40.957742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:40.957731    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:26:41.518693 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:41.518658    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:41.972129 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:41.972041    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" event={"ID":"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd","Type":"ContainerStarted","Data":"f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396"}
Oct 31 04:26:42.196022 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.195984    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:42.223472 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.223372    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:42.224460 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.224428    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 31 04:26:42.224628 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.224481    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 31 04:26:42.367978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.367927    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:42.509294 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.509259    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:42.975342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:42.975250    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:43.130557 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:43.130524    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:43.223541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:43.223498    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 31 04:26:43.223773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:43.223570    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 31 04:26:43.271915 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:43.271886    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:44.223117 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:44.223076    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 31 04:26:44.223744 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:44.223708    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 31 04:26:45.222821 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:45.222784    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 31 04:26:45.223041 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:45.222851    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 31 04:26:45.706516 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:45.706469    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:45.812569 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:45.812522    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:46.223071 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:46.223031    5906 patch_prober.go:29] interesting pod/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 31 04:26:46.223334 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:46.223090    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 31 04:26:46.699856 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:46.699824    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:47.227841 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:47.227804    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:47.229062 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:47.229035    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:26:51.292003 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:51.291949    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:26:55.795640 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:26:55.795602    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:00.303425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:00.303384    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:04.792341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:04.792301    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:05.846516 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:05.846461    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:06.381655 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:27:06.381610    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:27:06.697415 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:06.697337    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:09.308697 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:09.308660    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:11.893743 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:11.893707    5906 scope.go:122] "RemoveContainer" containerID="40eb1534a1314cd9c8cc163c0495249e12144afa692e494ccbfa926dca5ab55b"
Oct 31 04:27:11.902476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:11.902456    5906 scope.go:122] "RemoveContainer" containerID="005ba37a61cdcfe0a3b581054e90e79f9f94baa30f1734e29a526cf9f4447b58"
Oct 31 04:27:11.911398 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:11.911376    5906 scope.go:122] "RemoveContainer" containerID="84574e1ca95feee6a38a664e7204ef33cbd99aa674f1ac8a1c295ef91da2e925"
Oct 31 04:27:13.813097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:13.813058    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:18.392972 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:18.392934    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:22.903408 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:22.903372    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:27.407101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:27.407063    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:31.911314 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:31.911276    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:33.892134 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:33.892096    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:36.384967 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:27:36.384927    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:27:36.390345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:36.390317    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:40.907424 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:40.907376    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:45.406407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:45.406370    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:49.898427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:49.898348    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:54.414145 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:54.414105    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:27:58.895131 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:27:58.895097    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:02.606336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:02.606297    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:03.541755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:03.541714    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:06.380673 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:28:06.380636    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:28:07.638221 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:07.638185    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:08.446654 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:08.446614    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:12.995438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:12.995403    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:14.803865 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:14.803835    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:14.941647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:14.941611    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:17.413254 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:17.413188    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:19.075391 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:19.075359    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:19.215489 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:19.215459    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:21.249341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:21.249305    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"]
Oct 31 04:28:21.249843 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:21.249570    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" podUID="9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" containerName="maas-api" containerID="cri-o://a23853e2e5eaf249daa3ef4c6a5b45f13d68136416218f33449d29a51024ce85" gracePeriod=30
Oct 31 04:28:21.336377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:21.336344    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:22.502968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.502936    5906 generic.go:356] "Generic (PLEG): container finished" podID="9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" containerID="a23853e2e5eaf249daa3ef4c6a5b45f13d68136416218f33449d29a51024ce85" exitCode=0
Oct 31 04:28:22.503318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.503007    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" event={"ID":"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4","Type":"ContainerDied","Data":"a23853e2e5eaf249daa3ef4c6a5b45f13d68136416218f33449d29a51024ce85"}
Oct 31 04:28:22.503318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.503057    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw" event={"ID":"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4","Type":"ContainerDied","Data":"809187a9cf7276e175b455f6543a22f727016bce422467704ac0f5b755b8a17a"}
Oct 31 04:28:22.503318 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.503075    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="809187a9cf7276e175b455f6543a22f727016bce422467704ac0f5b755b8a17a"
Oct 31 04:28:22.508031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.508011    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:28:22.582697 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.582606    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs\") pod \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") "
Oct 31 04:28:22.582697 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.582662    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\") pod \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") "
Oct 31 04:28:22.582948 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.582786    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca\") pod \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") "
Oct 31 04:28:22.582948 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.582887    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6\" (UniqueName: \"kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6\") pod \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\" (UID: \"9cb6c7be-cf47-4c24-b5d4-88d683adc7b4\") "
Oct 31 04:28:22.583365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.583225    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca" pod "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" (UID: "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:28:22.583365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.583332    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:22.585606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.585578    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls" pod "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" (UID: "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:28:22.585606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.585595    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs" pod "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" (UID: "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:28:22.585752 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.585653    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6" pod "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" (UID: "9cb6c7be-cf47-4c24-b5d4-88d683adc7b4"). InnerVolumeSpecName "kube-api-access-vdvc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:28:22.684390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.684355    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:22.684390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.684384    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:22.684390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.684394    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdvc6\" (UniqueName: \"kubernetes.io/projected/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4-kube-api-access-vdvc6\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:22.778570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:22.778537    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:23.067691 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:23.067659    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:23.508270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:23.508241    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"
Oct 31 04:28:23.677607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:23.677572    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"]
Oct 31 04:28:23.729276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:23.729243    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-b097c5-56576765dd-5qmgw"]
Oct 31 04:28:24.351127 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:24.351094    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" path="/var/lib/kubelet/pods/9cb6c7be-cf47-4c24-b5d4-88d683adc7b4/volumes"
Oct 31 04:28:29.051838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:29.051807    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"]
Oct 31 04:28:29.068335 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:29.068295    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-g6kt2"]
Oct 31 04:28:30.351402 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:30.351366    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7ca5118-5788-44f1-a5c4-9beea2a6a49a" path="/var/lib/kubelet/pods/b7ca5118-5788-44f1-a5c4-9beea2a6a49a/volumes"
Oct 31 04:28:33.833411 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:33.833377    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:34.822299 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:34.822255    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:36.381977 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:28:36.381938    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:28:39.715251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:39.715200    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:39.852978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:39.852946    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:40.077676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:40.077637    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:40.080169 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:40.080137    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"]
Oct 31 04:28:40.080641 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:40.080609    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy" containerID="cri-o://f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396" gracePeriod=30
Oct 31 04:28:44.442450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:44.442414    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:44.765340 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:44.765299    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:28:45.347048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.346976    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:28:45.443493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443455    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443508    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443531    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443565    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443613    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443700    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443744    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443765    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443774    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:45.443977 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.443797    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data\") pod \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\" (UID: \"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd\") "
Oct 31 04:28:45.444535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.444172    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.444535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.444234    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:45.444535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.444283    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:28:45.444535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.444379    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:45.444535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.444438    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:45.445886 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.445856    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:28:45.446193 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.446161    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:28:45.446336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.446274    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:45.446336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.446276    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n" pod "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" (UID: "c93a6a3d-fd40-4238-b9ea-ea4cf75626cd"). InnerVolumeSpecName "kube-api-access-v9d4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:28:45.544912 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544874    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9d4n\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-kube-api-access-v9d4n\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.544912 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544908    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.544912 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544917    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.545196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544926    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.545196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544935    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.545196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544943    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.545196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544952    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.545196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.544960    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:45.617130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.617042    5906 generic.go:356] "Generic (PLEG): container finished" podID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerID="f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396" exitCode=0
Oct 31 04:28:45.617130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.617095    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" event={"ID":"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd","Type":"ContainerDied","Data":"f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396"}
Oct 31 04:28:45.617130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.617102    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"
Oct 31 04:28:45.617477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.617138    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b" event={"ID":"c93a6a3d-fd40-4238-b9ea-ea4cf75626cd","Type":"ContainerDied","Data":"8258638002d9e301509ca8fd2b3a6a9012bc29a92242cd9c40b91d461d5a03bb"}
Oct 31 04:28:45.617477 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.617160    5906 scope.go:122] "RemoveContainer" containerID="f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396"
Oct 31 04:28:45.626820 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.626796    5906 scope.go:122] "RemoveContainer" containerID="6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"
Oct 31 04:28:45.635539 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.635518    5906 scope.go:122] "RemoveContainer" containerID="f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396"
Oct 31 04:28:45.635855 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:28:45.635834    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396\": container with ID starting with f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396 not found: ID does not exist" containerID="f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396"
Oct 31 04:28:45.635904 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.635868    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396"} err="failed to get container status \"f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396\": rpc error: code = NotFound desc = could not find container \"f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396\": container with ID starting with f31340cb965df31668c45c0013c3a92e45837e23a60647cf84f9e29ba8cc0396 not found: ID does not exist"
Oct 31 04:28:45.635904 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.635887    5906 scope.go:122] "RemoveContainer" containerID="6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"
Oct 31 04:28:45.636171 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:28:45.636149    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71\": container with ID starting with 6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71 not found: ID does not exist" containerID="6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"
Oct 31 04:28:45.636255 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.636176    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71"} err="failed to get container status \"6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71\": rpc error: code = NotFound desc = could not find container \"6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71\": container with ID starting with 6cb0c73c2d6917a625120f7e45bd9331c707a8beed4e158326ce736613179b71 not found: ID does not exist"
Oct 31 04:28:45.788199 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.788156    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"]
Oct 31 04:28:45.790341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:45.790311    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-b097c5-openshift-default-797d95c7dfqjs2b"]
Oct 31 04:28:46.352097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:46.352061    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" path="/var/lib/kubelet/pods/c93a6a3d-fd40-4238-b9ea-ea4cf75626cd/volumes"
Oct 31 04:28:50.372279 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:28:50.372237    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:04.486048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:04.486011    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:05.846312 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:05.846273    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:06.380973 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:29:06.380938    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:29:13.253380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.253336    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-4wtlq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:29:13.254240 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.254184    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:29:13.317431 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.317380    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-zcwhh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:29:13.317788 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.317755    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:29:13.360386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.360345    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-rx2xb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:29:13.360713 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.360664    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:29:13.379874 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.379829    5906 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-64ln7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:29:13.380182 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:13.380153    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.139:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:29:20.867458 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:20.867373    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:21.004916 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:21.004881    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:22.623214 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:22.623168    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:22.763154 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:22.763120    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:24.856342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:24.856306    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:24.994629 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:24.994588    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:30.091466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:30.091426    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:34.631893 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:29:34.631856    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:29:36.542231 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:29:36.541411    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:30:00.168857 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.168822    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"]
Oct 31 04:30:00.169385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169279    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy"
Oct 31 04:30:00.169385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169293    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy"
Oct 31 04:30:00.169385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169308    5906 cpu_manager.go:414] "containerMap: removing container" podUID="9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" containerName="maas-api"
Oct 31 04:30:00.169385 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169313    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" containerName="maas-api"
Oct 31 04:30:00.169576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169406    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy"
Oct 31 04:30:00.169576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169421    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy"
Oct 31 04:30:00.169576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.169429    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="9cb6c7be-cf47-4c24-b5d4-88d683adc7b4" containerName="maas-api"
Oct 31 04:30:00.173320 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.173296    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.228876 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.228826    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"]
Oct 31 04:30:00.264425 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.264393    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890350-79rln\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.264601 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.264433    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr6c6\" (UniqueName: \"kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6\") pod \"maas-api-key-cleanup-29890350-79rln\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.278278 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.278246    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:30:00.365593 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.365555    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890350-79rln\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.365776 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.365600    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr6c6\" (UniqueName: \"kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6\") pod \"maas-api-key-cleanup-29890350-79rln\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.366256 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.366236    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca\") pod \"maas-api-key-cleanup-29890350-79rln\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.429143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.429069    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr6c6\" (UniqueName: \"kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6\") pod \"maas-api-key-cleanup-29890350-79rln\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.486939 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.486894    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:00.860970 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:30:00.860932    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e21d1d7_9621_4f7f_ac69_dc737d149194.slice/crio-7c151c4d940489cfd54ff9150d2acf00639d315852bce9494487ce2542c7b4cc WatchSource:0}: Error finding container 7c151c4d940489cfd54ff9150d2acf00639d315852bce9494487ce2542c7b4cc: Status 404 returned error can't find the container with id 7c151c4d940489cfd54ff9150d2acf00639d315852bce9494487ce2542c7b4cc
Oct 31 04:30:00.862716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:00.862687    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"]
Oct 31 04:30:01.070647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:01.070609    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" event={"ID":"0e21d1d7-9621-4f7f-ac69-dc737d149194","Type":"ContainerStarted","Data":"7c151c4d940489cfd54ff9150d2acf00639d315852bce9494487ce2542c7b4cc"}
Oct 31 04:30:02.077552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:02.077516    5906 generic.go:356] "Generic (PLEG): container finished" podID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerID="a088475dacb9c990740b7fd3826b821f16ad7485d1170a7e64486b88a9f51818" exitCode=60
Oct 31 04:30:02.078053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:02.077588    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" event={"ID":"0e21d1d7-9621-4f7f-ac69-dc737d149194","Type":"ContainerDied","Data":"a088475dacb9c990740b7fd3826b821f16ad7485d1170a7e64486b88a9f51818"}
Oct 31 04:30:02.078053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:02.077848    5906 scope.go:122] "RemoveContainer" containerID="a088475dacb9c990740b7fd3826b821f16ad7485d1170a7e64486b88a9f51818"
Oct 31 04:30:03.085341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:03.085304    5906 generic.go:356] "Generic (PLEG): container finished" podID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerID="2d51e1408cb5739e41e7b4d4f7dce9c6ee33ae9a8664eefac48c26ffac04a044" exitCode=60
Oct 31 04:30:03.085825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:03.085403    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" event={"ID":"0e21d1d7-9621-4f7f-ac69-dc737d149194","Type":"ContainerDied","Data":"2d51e1408cb5739e41e7b4d4f7dce9c6ee33ae9a8664eefac48c26ffac04a044"}
Oct 31 04:30:03.085825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:03.085476    5906 scope.go:122] "RemoveContainer" containerID="a088475dacb9c990740b7fd3826b821f16ad7485d1170a7e64486b88a9f51818"
Oct 31 04:30:03.085825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:03.085751    5906 scope.go:122] "RemoveContainer" containerID="2d51e1408cb5739e41e7b4d4f7dce9c6ee33ae9a8664eefac48c26ffac04a044"
Oct 31 04:30:03.086055 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:30:03.086030    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890350-79rln_odh-ai-gateway-infra(0e21d1d7-9621-4f7f-ac69-dc737d149194)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194"
Oct 31 04:30:04.092300 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:04.092274    5906 scope.go:122] "RemoveContainer" containerID="2d51e1408cb5739e41e7b4d4f7dce9c6ee33ae9a8664eefac48c26ffac04a044"
Oct 31 04:30:04.092693 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:30:04.092513    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29890350-79rln_odh-ai-gateway-infra(0e21d1d7-9621-4f7f-ac69-dc737d149194)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194"
Oct 31 04:30:06.381731 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:30:06.381694    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:30:12.037735 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:12.037703    5906 scope.go:122] "RemoveContainer" containerID="7d6a568c3a31fe1d231941c9b167362952124eb6ce11ff390200b45c18501863"
Oct 31 04:30:17.345181 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:17.345147    5906 scope.go:122] "RemoveContainer" containerID="2d51e1408cb5739e41e7b4d4f7dce9c6ee33ae9a8664eefac48c26ffac04a044"
Oct 31 04:30:18.183754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:18.183723    5906 generic.go:356] "Generic (PLEG): container finished" podID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerID="26ffe83718cc8050d980f17deb390145cb27972d0ee5196df482bc303461f52a" exitCode=60
Oct 31 04:30:18.183958 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:18.183799    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" event={"ID":"0e21d1d7-9621-4f7f-ac69-dc737d149194","Type":"ContainerDied","Data":"26ffe83718cc8050d980f17deb390145cb27972d0ee5196df482bc303461f52a"}
Oct 31 04:30:18.183958 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:18.183872    5906 scope.go:122] "RemoveContainer" containerID="2d51e1408cb5739e41e7b4d4f7dce9c6ee33ae9a8664eefac48c26ffac04a044"
Oct 31 04:30:18.184336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:18.184318    5906 scope.go:122] "RemoveContainer" containerID="26ffe83718cc8050d980f17deb390145cb27972d0ee5196df482bc303461f52a"
Oct 31 04:30:18.184607 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:30:18.184585    5906 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29890350-79rln_odh-ai-gateway-infra(0e21d1d7-9621-4f7f-ac69-dc737d149194)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194"
Oct 31 04:30:19.311780 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.311742    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"]
Oct 31 04:30:19.464262 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.464234    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:19.522200 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.522169    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca\") pod \"0e21d1d7-9621-4f7f-ac69-dc737d149194\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") "
Oct 31 04:30:19.522408 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.522278    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6\" (UniqueName: \"kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6\") pod \"0e21d1d7-9621-4f7f-ac69-dc737d149194\" (UID: \"0e21d1d7-9621-4f7f-ac69-dc737d149194\") "
Oct 31 04:30:19.522552 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.522528    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca" pod "0e21d1d7-9621-4f7f-ac69-dc737d149194" (UID: "0e21d1d7-9621-4f7f-ac69-dc737d149194"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:30:19.522607 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.522593    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e21d1d7-9621-4f7f-ac69-dc737d149194-maas-api-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:19.524372 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.524350    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6" pod "0e21d1d7-9621-4f7f-ac69-dc737d149194" (UID: "0e21d1d7-9621-4f7f-ac69-dc737d149194"). InnerVolumeSpecName "kube-api-access-qr6c6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:30:19.624149 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:19.624061    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qr6c6\" (UniqueName: \"kubernetes.io/projected/0e21d1d7-9621-4f7f-ac69-dc737d149194-kube-api-access-qr6c6\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:20.196520 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:20.196484    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"
Oct 31 04:30:20.196725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:20.196523    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln" event={"ID":"0e21d1d7-9621-4f7f-ac69-dc737d149194","Type":"ContainerDied","Data":"7c151c4d940489cfd54ff9150d2acf00639d315852bce9494487ce2542c7b4cc"}
Oct 31 04:30:20.196725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:20.196583    5906 scope.go:122] "RemoveContainer" containerID="26ffe83718cc8050d980f17deb390145cb27972d0ee5196df482bc303461f52a"
Oct 31 04:30:20.365570 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:20.365505    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"]
Oct 31 04:30:20.416926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:20.416888    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29890350-79rln"]
Oct 31 04:30:22.351405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:22.351367    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" path="/var/lib/kubelet/pods/0e21d1d7-9621-4f7f-ac69-dc737d149194/volumes"
Oct 31 04:30:25.017573 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:25.017542    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:30:26.047853 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:26.047814    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"]
Oct 31 04:30:26.099183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:26.099148    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-lhl7k"]
Oct 31 04:30:26.350937 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:26.350851    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45b7e959-2762-496e-8826-fa4460254b35" path="/var/lib/kubelet/pods/45b7e959-2762-496e-8826-fa4460254b35/volumes"
Oct 31 04:30:31.069147 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:31.069052    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:30:34.920473 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:34.920426    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:30:36.529569 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:30:36.529511    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:30:40.034867 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:40.034793    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:30:44.238260 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.238197    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"]
Oct 31 04:30:44.239559 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.239530    5906 cpu_manager.go:414] "containerMap: removing container" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.239736 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.239721    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.239856 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.239844    5906 cpu_manager.go:414] "containerMap: removing container" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.239949 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.239937    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.240048 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240038    5906 cpu_manager.go:414] "containerMap: removing container" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.240128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240115    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.240279 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240264    5906 cpu_manager.go:414] "containerMap: removing container" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy"
Oct 31 04:30:44.240376 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240364    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c93a6a3d-fd40-4238-b9ea-ea4cf75626cd" containerName="istio-proxy"
Oct 31 04:30:44.240587 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240572    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.240694 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240683    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.240790 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.240780    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e21d1d7-9621-4f7f-ac69-dc737d149194" containerName="cleanup"
Oct 31 04:30:44.270572 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.270530    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.300934 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.292997    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"]
Oct 31 04:30:44.342353 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.342292    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:30:44.342353 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.342346    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:30:44.411943 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.411904    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.434377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.434334    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkfrl\" (UniqueName: \"kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.434806 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.434780    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.538073 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.538034    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.539264 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.539238    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkfrl\" (UniqueName: \"kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.539527 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.539507    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.539825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.539140    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.558230 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.553774    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.605392 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.605323    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkfrl\" (UniqueName: \"kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl\") pod \"maas-api-revoke-sub-3e31a506-8qn7j\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:44.660455 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:44.660414    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:45.506231 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:45.494458    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"]
Oct 31 04:30:46.602195 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:46.602160    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j" event={"ID":"b001c13e-328d-4598-9a8c-43727b8b9837","Type":"ContainerStarted","Data":"4b49825b8afa67039937c149a4378e6edc71db38def237b87126d923a6c54386"}
Oct 31 04:30:46.602716 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:46.602224    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j" event={"ID":"b001c13e-328d-4598-9a8c-43727b8b9837","Type":"ContainerStarted","Data":"a0ae2a7e1fd0670062c2c5c6b5628b1385313f349003879538cda03f7a0c8916"}
Oct 31 04:30:46.721725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:46.721672    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j" podStartSLOduration=2.721657684 podStartE2EDuration="2.721657684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:30:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:30:46.720554857 +0000 UTC m=+3696.554415162" watchObservedRunningTime="2026-10-31 04:30:46.721657684 +0000 UTC m=+3696.555517989"
Oct 31 04:30:47.609025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:47.608984    5906 generic.go:356] "Generic (PLEG): container finished" podID="b001c13e-328d-4598-9a8c-43727b8b9837" containerID="4b49825b8afa67039937c149a4378e6edc71db38def237b87126d923a6c54386" exitCode=0
Oct 31 04:30:47.609461 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:47.609068    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j" event={"ID":"b001c13e-328d-4598-9a8c-43727b8b9837","Type":"ContainerDied","Data":"4b49825b8afa67039937c149a4378e6edc71db38def237b87126d923a6c54386"}
Oct 31 04:30:48.755283 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.755258    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:48.772225 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.772177    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca\") pod \"b001c13e-328d-4598-9a8c-43727b8b9837\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") "
Oct 31 04:30:48.772395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.772249    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl\" (UniqueName: \"kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl\") pod \"b001c13e-328d-4598-9a8c-43727b8b9837\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") "
Oct 31 04:30:48.772395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.772328    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca\") pod \"b001c13e-328d-4598-9a8c-43727b8b9837\" (UID: \"b001c13e-328d-4598-9a8c-43727b8b9837\") "
Oct 31 04:30:48.772690 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.772666    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca" pod "b001c13e-328d-4598-9a8c-43727b8b9837" (UID: "b001c13e-328d-4598-9a8c-43727b8b9837"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:30:48.774370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.774344    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca" pod "b001c13e-328d-4598-9a8c-43727b8b9837" (UID: "b001c13e-328d-4598-9a8c-43727b8b9837"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:30:48.774478 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.774368    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl" pod "b001c13e-328d-4598-9a8c-43727b8b9837" (UID: "b001c13e-328d-4598-9a8c-43727b8b9837"). InnerVolumeSpecName "kube-api-access-hkfrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:30:48.873719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.873636    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:48.873719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.873668    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkfrl\" (UniqueName: \"kubernetes.io/projected/b001c13e-328d-4598-9a8c-43727b8b9837-kube-api-access-hkfrl\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:48.873719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:48.873679    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b001c13e-328d-4598-9a8c-43727b8b9837-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:49.619858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:49.619823    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j" event={"ID":"b001c13e-328d-4598-9a8c-43727b8b9837","Type":"ContainerDied","Data":"a0ae2a7e1fd0670062c2c5c6b5628b1385313f349003879538cda03f7a0c8916"}
Oct 31 04:30:49.619858 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:49.619855    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"
Oct 31 04:30:49.620088 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:49.619897    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0ae2a7e1fd0670062c2c5c6b5628b1385313f349003879538cda03f7a0c8916"
Oct 31 04:30:54.050053 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:54.050018    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"]
Oct 31 04:30:54.101261 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:54.101227    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-l2gwb"]
Oct 31 04:30:54.351402 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:30:54.351325    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3e57001-e010-4cba-94e3-6eabe4da8e9e" path="/var/lib/kubelet/pods/b3e57001-e010-4cba-94e3-6eabe4da8e9e/volumes"
Oct 31 04:31:06.381931 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:31:06.381892    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:31:13.810855 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.810817    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"]
Oct 31 04:31:13.811299 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.811283    5906 cpu_manager.go:414] "containerMap: removing container" podUID="b001c13e-328d-4598-9a8c-43727b8b9837" containerName="revoke-keys"
Oct 31 04:31:13.811299 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.811296    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b001c13e-328d-4598-9a8c-43727b8b9837" containerName="revoke-keys"
Oct 31 04:31:13.811402 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.811392    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="b001c13e-328d-4598-9a8c-43727b8b9837" containerName="revoke-keys"
Oct 31 04:31:13.814530 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.814513    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:13.869755 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.869715    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"]
Oct 31 04:31:13.909742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.909702    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:13.909929 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.909785    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj6jt\" (UniqueName: \"kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:13.909929 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.909844    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:13.919614 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.919589    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 31 04:31:13.919714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:13.919662    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:31:14.010721 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.010684    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.010946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.010756    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj6jt\" (UniqueName: \"kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.010946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.010784    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.011475 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.011447    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.013325 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.013301    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.073855 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.073781    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj6jt\" (UniqueName: \"kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt\") pod \"maas-api-revoke-sub-e3a0f9d5-htj7m\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.127118 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.127083    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:14.496472 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:31:14.496429    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab887103_bf92_47b2_8f92_bb112d545045.slice/crio-ab322256c53babd05ea4ddc20acb8f754197b6aa54b2bb469e3a4116ce564339 WatchSource:0}: Error finding container ab322256c53babd05ea4ddc20acb8f754197b6aa54b2bb469e3a4116ce564339: Status 404 returned error can't find the container with id ab322256c53babd05ea4ddc20acb8f754197b6aa54b2bb469e3a4116ce564339
Oct 31 04:31:14.497919 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.497893    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"]
Oct 31 04:31:14.795032 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.794993    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m" event={"ID":"ab887103-bf92-47b2-8f92-bb112d545045","Type":"ContainerStarted","Data":"404ca826066241960fbbc340e7acf7aca0983697625d2fa547de065188c88d65"}
Oct 31 04:31:14.795224 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.795057    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m" event={"ID":"ab887103-bf92-47b2-8f92-bb112d545045","Type":"ContainerStarted","Data":"ab322256c53babd05ea4ddc20acb8f754197b6aa54b2bb469e3a4116ce564339"}
Oct 31 04:31:14.914647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:14.914594    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m" podStartSLOduration=1.914581192 podStartE2EDuration="1.914581192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:31:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:31:14.914071566 +0000 UTC m=+3724.747931865" watchObservedRunningTime="2026-10-31 04:31:14.914581192 +0000 UTC m=+3724.748441493"
Oct 31 04:31:15.801728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:15.801690    5906 generic.go:356] "Generic (PLEG): container finished" podID="ab887103-bf92-47b2-8f92-bb112d545045" containerID="404ca826066241960fbbc340e7acf7aca0983697625d2fa547de065188c88d65" exitCode=0
Oct 31 04:31:15.801922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:15.801775    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m" event={"ID":"ab887103-bf92-47b2-8f92-bb112d545045","Type":"ContainerDied","Data":"404ca826066241960fbbc340e7acf7aca0983697625d2fa547de065188c88d65"}
Oct 31 04:31:16.945839 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:16.945807    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:17.043360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.043323    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca\") pod \"ab887103-bf92-47b2-8f92-bb112d545045\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") "
Oct 31 04:31:17.043565 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.043546    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt\" (UniqueName: \"kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt\") pod \"ab887103-bf92-47b2-8f92-bb112d545045\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") "
Oct 31 04:31:17.043619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.043598    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca\") pod \"ab887103-bf92-47b2-8f92-bb112d545045\" (UID: \"ab887103-bf92-47b2-8f92-bb112d545045\") "
Oct 31 04:31:17.043773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.043753    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca" pod "ab887103-bf92-47b2-8f92-bb112d545045" (UID: "ab887103-bf92-47b2-8f92-bb112d545045"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:31:17.043950 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.043936    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-openshift-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:17.045752 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.045723    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca" pod "ab887103-bf92-47b2-8f92-bb112d545045" (UID: "ab887103-bf92-47b2-8f92-bb112d545045"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:31:17.045879 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.045840    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt" pod "ab887103-bf92-47b2-8f92-bb112d545045" (UID: "ab887103-bf92-47b2-8f92-bb112d545045"). InnerVolumeSpecName "kube-api-access-jj6jt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:31:17.144975 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.144885    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jj6jt\" (UniqueName: \"kubernetes.io/projected/ab887103-bf92-47b2-8f92-bb112d545045-kube-api-access-jj6jt\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:17.144975 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.144915    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab887103-bf92-47b2-8f92-bb112d545045-maas-api-xks-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:17.812877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.812849    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"
Oct 31 04:31:17.812877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.812858    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m" event={"ID":"ab887103-bf92-47b2-8f92-bb112d545045","Type":"ContainerDied","Data":"ab322256c53babd05ea4ddc20acb8f754197b6aa54b2bb469e3a4116ce564339"}
Oct 31 04:31:17.813096 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:17.812899    5906 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab322256c53babd05ea4ddc20acb8f754197b6aa54b2bb469e3a4116ce564339"
Oct 31 04:31:27.100861 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.100823    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-wffcm"]
Oct 31 04:31:27.101377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.101361    5906 cpu_manager.go:414] "containerMap: removing container" podUID="ab887103-bf92-47b2-8f92-bb112d545045" containerName="revoke-keys"
Oct 31 04:31:27.101377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.101378    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab887103-bf92-47b2-8f92-bb112d545045" containerName="revoke-keys"
Oct 31 04:31:27.101467 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.101462    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab887103-bf92-47b2-8f92-bb112d545045" containerName="revoke-keys"
Oct 31 04:31:27.104887 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.104865    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.114813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.114784    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c8b55458c-887dn"]
Oct 31 04:31:27.119613 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.119589    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.164836 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.164803    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 04:31:27.165005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.164883    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 31 04:31:27.165005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.164924    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gc447\""
Oct 31 04:31:27.212630 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.212590    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-wffcm"]
Oct 31 04:31:27.252036 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252005    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr9bg\" (UniqueName: \"kubernetes.io/projected/a3475cd3-fd85-4b01-8e83-b449bd11a898-kube-api-access-hr9bg\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.252283 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252069    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3475cd3-fd85-4b01-8e83-b449bd11a898-maas-api-tls\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.252283 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252167    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.252283 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252237    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6sk5\" (UniqueName: \"kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.252462 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252309    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.252462 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252382    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.252462 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252399    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3475cd3-fd85-4b01-8e83-b449bd11a898-metrics-certs\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.252462 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.252449    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3475cd3-fd85-4b01-8e83-b449bd11a898-openshift-service-ca\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.261490 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.261460    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c8b55458c-887dn"]
Oct 31 04:31:27.353451 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353372    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6sk5\" (UniqueName: \"kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.353451 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353406    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.353725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353459    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.353725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353479    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3475cd3-fd85-4b01-8e83-b449bd11a898-metrics-certs\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.353725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353499    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3475cd3-fd85-4b01-8e83-b449bd11a898-openshift-service-ca\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.353725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353668    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr9bg\" (UniqueName: \"kubernetes.io/projected/a3475cd3-fd85-4b01-8e83-b449bd11a898-kube-api-access-hr9bg\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.353945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353794    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3475cd3-fd85-4b01-8e83-b449bd11a898-maas-api-tls\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.353945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.353831    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.354309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.354281    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3475cd3-fd85-4b01-8e83-b449bd11a898-openshift-service-ca\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.354421 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.354287    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.356148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.356118    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.356266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.356125    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3475cd3-fd85-4b01-8e83-b449bd11a898-metrics-certs\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.356266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.356237    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3475cd3-fd85-4b01-8e83-b449bd11a898-maas-api-tls\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.356646 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.356625    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.416658 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.416626    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr9bg\" (UniqueName: \"kubernetes.io/projected/a3475cd3-fd85-4b01-8e83-b449bd11a898-kube-api-access-hr9bg\") pod \"maas-api-c8b55458c-887dn\" (UID: \"a3475cd3-fd85-4b01-8e83-b449bd11a898\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.416827 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.416759    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6sk5\" (UniqueName: \"kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5\") pod \"maas-api-68c9679795-wffcm\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.419438 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.419418    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:27.435094 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.435067    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:27.811881 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:31:27.811849    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f643f54_be71_467e_bcb2_086e01c7efe7.slice/crio-fc0786200ad88f150322ffa9f8e44f8e503b15dcdae175dd96e2131edbd732ab WatchSource:0}: Error finding container fc0786200ad88f150322ffa9f8e44f8e503b15dcdae175dd96e2131edbd732ab: Status 404 returned error can't find the container with id fc0786200ad88f150322ffa9f8e44f8e503b15dcdae175dd96e2131edbd732ab
Oct 31 04:31:27.813545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.813520    5906 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:31:27.814002 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.813971    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-wffcm"]
Oct 31 04:31:27.847375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.847340    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c8b55458c-887dn"]
Oct 31 04:31:27.852882 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:31:27.852835    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3475cd3_fd85_4b01_8e83_b449bd11a898.slice/crio-732a24fb573f1c24c14735cb0a4789360a8a1a15f7ce2d5a110d49a31bfbb3eb WatchSource:0}: Error finding container 732a24fb573f1c24c14735cb0a4789360a8a1a15f7ce2d5a110d49a31bfbb3eb: Status 404 returned error can't find the container with id 732a24fb573f1c24c14735cb0a4789360a8a1a15f7ce2d5a110d49a31bfbb3eb
Oct 31 04:31:27.863668 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.863623    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn" event={"ID":"a3475cd3-fd85-4b01-8e83-b449bd11a898","Type":"ContainerStarted","Data":"732a24fb573f1c24c14735cb0a4789360a8a1a15f7ce2d5a110d49a31bfbb3eb"}
Oct 31 04:31:27.864844 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:27.864819    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm" event={"ID":"7f643f54-be71-467e-bcb2-086e01c7efe7","Type":"ContainerStarted","Data":"fc0786200ad88f150322ffa9f8e44f8e503b15dcdae175dd96e2131edbd732ab"}
Oct 31 04:31:29.876750 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:29.876703    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm" event={"ID":"7f643f54-be71-467e-bcb2-086e01c7efe7","Type":"ContainerStarted","Data":"606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30"}
Oct 31 04:31:29.877481 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:29.876815    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:29.878342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:29.878321    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn" event={"ID":"a3475cd3-fd85-4b01-8e83-b449bd11a898","Type":"ContainerStarted","Data":"3f86743be092fb6d7235559859b81fdbad475c58d1746f5b3ba27535e341d993"}
Oct 31 04:31:29.878458 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:29.878446    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:29.998409 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:29.998351    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm" podStartSLOduration=1.998641193 podStartE2EDuration="2.998338002s" totalImagesPullingTime="999.696809ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:31:29.997490479 +0000 UTC m=+3739.831350805" watchObservedRunningTime="2026-10-31 04:31:29.998338002 +0000 UTC m=+3739.832198300"
Oct 31 04:31:30.120494 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:30.120436    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn" podStartSLOduration=2.136778154 podStartE2EDuration="3.12042175s" totalImagesPullingTime="983.643596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:31:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:31:30.118463262 +0000 UTC m=+3739.952323561" watchObservedRunningTime="2026-10-31 04:31:30.12042175 +0000 UTC m=+3739.954282127"
Oct 31 04:31:33.557133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:33.557099    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:34.087813 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.087777    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"]
Oct 31 04:31:34.091686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.091670    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.145884 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.145829    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-40966d85-openshift-default-dockercfg-45wz4\""
Oct 31 04:31:34.181432 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.181405    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:34.199453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.199424    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"]
Oct 31 04:31:34.227377 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227339    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227386    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227501    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f64d6\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227583 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227534    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227634    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227728 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227689    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227815 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227767    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227863 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227820    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.227908 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.227888    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328627 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328589    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f64d6\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328627 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328637    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328670    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328693    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328750    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328800    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.328888 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328845    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328893    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.328917    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329183 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.329171    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329326 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.329284    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329537 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.329515    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329592 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.329520    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.329772 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.329747    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.331404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.331380    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.331404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.331392    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.396174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.396081    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f64d6\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.396504 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.396479    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token\") pod \"e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") " pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.404429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.404397    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:34.446854 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.446763    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:34.850110 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:31:34.850077    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b524a97_36a2_4669_a250_566dace2e8ab.slice/crio-4da2cca7953bf531aad3295e5e42b1bfcb1515791c6179f67bfd83503abf0de9 WatchSource:0}: Error finding container 4da2cca7953bf531aad3295e5e42b1bfcb1515791c6179f67bfd83503abf0de9: Status 404 returned error can't find the container with id 4da2cca7953bf531aad3295e5e42b1bfcb1515791c6179f67bfd83503abf0de9
Oct 31 04:31:34.851699 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.851673    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"]
Oct 31 04:31:34.852459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.852424    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:31:34.852581 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.852514    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:31:34.852581 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.852560    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:31:34.907600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:34.907570    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" event={"ID":"5b524a97-36a2-4669-a250-566dace2e8ab","Type":"ContainerStarted","Data":"4da2cca7953bf531aad3295e5e42b1bfcb1515791c6179f67bfd83503abf0de9"}
Oct 31 04:31:35.267065 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:35.267031    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:35.889221 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:35.889188    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:35.889676 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:35.889579    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c8b55458c-887dn"
Oct 31 04:31:35.915107 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:35.915066    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" event={"ID":"5b524a97-36a2-4669-a250-566dace2e8ab","Type":"ContainerStarted","Data":"2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"}
Oct 31 04:31:36.140841 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.140736    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podStartSLOduration=2.140722302 podStartE2EDuration="2.140722302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:31:36.139684177 +0000 UTC m=+3745.973544481" watchObservedRunningTime="2026-10-31 04:31:36.140722302 +0000 UTC m=+3745.974582606"
Oct 31 04:31:36.294864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.294823    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-wffcm"]
Oct 31 04:31:36.295068 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.295019    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm" podUID="7f643f54-be71-467e-bcb2-086e01c7efe7" containerName="maas-api" containerID="cri-o://606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30" gracePeriod=30
Oct 31 04:31:36.383093 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:31:36.383052    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:31:36.405341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.405254    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:36.411049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.410960    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:36.547993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.547961    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:36.662345 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.662254    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs\") pod \"7f643f54-be71-467e-bcb2-086e01c7efe7\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") "
Oct 31 04:31:36.662535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.662363    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5\" (UniqueName: \"kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5\") pod \"7f643f54-be71-467e-bcb2-086e01c7efe7\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") "
Oct 31 04:31:36.662535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.662405    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls\") pod \"7f643f54-be71-467e-bcb2-086e01c7efe7\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") "
Oct 31 04:31:36.662672 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.662610    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca\") pod \"7f643f54-be71-467e-bcb2-086e01c7efe7\" (UID: \"7f643f54-be71-467e-bcb2-086e01c7efe7\") "
Oct 31 04:31:36.662974 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.662942    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca" pod "7f643f54-be71-467e-bcb2-086e01c7efe7" (UID: "7f643f54-be71-467e-bcb2-086e01c7efe7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:31:36.663141 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.663124    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f643f54-be71-467e-bcb2-086e01c7efe7-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:36.664606 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.664577    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls" pod "7f643f54-be71-467e-bcb2-086e01c7efe7" (UID: "7f643f54-be71-467e-bcb2-086e01c7efe7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:31:36.665031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.664719    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5" pod "7f643f54-be71-467e-bcb2-086e01c7efe7" (UID: "7f643f54-be71-467e-bcb2-086e01c7efe7"). InnerVolumeSpecName "kube-api-access-z6sk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:31:36.665031 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.664810    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs" pod "7f643f54-be71-467e-bcb2-086e01c7efe7" (UID: "7f643f54-be71-467e-bcb2-086e01c7efe7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:31:36.764081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.764039    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6sk5\" (UniqueName: \"kubernetes.io/projected/7f643f54-be71-467e-bcb2-086e01c7efe7-kube-api-access-z6sk5\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:36.764081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.764074    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:36.764081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.764087    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f643f54-be71-467e-bcb2-086e01c7efe7-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:36.921763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.921668    5906 generic.go:356] "Generic (PLEG): container finished" podID="7f643f54-be71-467e-bcb2-086e01c7efe7" containerID="606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30" exitCode=0
Oct 31 04:31:36.921763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.921720    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm"
Oct 31 04:31:36.921763 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.921750    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm" event={"ID":"7f643f54-be71-467e-bcb2-086e01c7efe7","Type":"ContainerDied","Data":"606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30"}
Oct 31 04:31:36.922374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.921798    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-wffcm" event={"ID":"7f643f54-be71-467e-bcb2-086e01c7efe7","Type":"ContainerDied","Data":"fc0786200ad88f150322ffa9f8e44f8e503b15dcdae175dd96e2131edbd732ab"}
Oct 31 04:31:36.922374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.921819    5906 scope.go:122] "RemoveContainer" containerID="606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30"
Oct 31 04:31:36.922374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.922234    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:36.923384 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.923362    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:31:36.931574 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.931557    5906 scope.go:122] "RemoveContainer" containerID="606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30"
Oct 31 04:31:36.931839 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:31:36.931821    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30\": container with ID starting with 606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30 not found: ID does not exist" containerID="606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30"
Oct 31 04:31:36.931894 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:36.931849    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30"} err="failed to get container status \"606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30\": rpc error: code = NotFound desc = could not find container \"606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30\": container with ID starting with 606adf7e9a6e1c254e4c71743e914197b306b129f89e11e2dd955d913ec4eb30 not found: ID does not exist"
Oct 31 04:31:37.216683 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:37.216586    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-wffcm"]
Oct 31 04:31:37.218963 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:37.218937    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-wffcm"]
Oct 31 04:31:38.251143 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:38.251104    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:38.351021 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:38.350980    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f643f54-be71-467e-bcb2-086e01c7efe7" path="/var/lib/kubelet/pods/7f643f54-be71-467e-bcb2-086e01c7efe7/volumes"
Oct 31 04:31:38.421627 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:38.421597    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:38.569529 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:38.569445    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:41.064990 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.064953    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"]
Oct 31 04:31:41.067383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.065413    5906 cpu_manager.go:414] "containerMap: removing container" podUID="7f643f54-be71-467e-bcb2-086e01c7efe7" containerName="maas-api"
Oct 31 04:31:41.067383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.065425    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f643f54-be71-467e-bcb2-086e01c7efe7" containerName="maas-api"
Oct 31 04:31:41.067383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.065525    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f643f54-be71-467e-bcb2-086e01c7efe7" containerName="maas-api"
Oct 31 04:31:41.068489 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.068470    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.105005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.104961    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.105005 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.105006    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkc2p\" (UniqueName: \"kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.105272 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.105139    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.105272 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.105221    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.180647 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.180615    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-40966d85-serving-cert\""
Oct 31 04:31:41.182599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.182576    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"]
Oct 31 04:31:41.206030 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.205985    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.206285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.206058    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.206285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.206094    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.206285 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.206116    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkc2p\" (UniqueName: \"kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.206285 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:31:41.206272    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-serving-cert: secret "maas-api-e2e-ait-40966d85-serving-cert" not found
Oct 31 04:31:41.206613 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:31:41.206350    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls podName:0ee89bf2-5092-4782-80c2-959ecee53134 nodeName:}" failed. No retries permitted until 2026-10-31 04:31:41.706329383 +0000 UTC m=+3751.540189687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls") pod "maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" (UID: "0ee89bf2-5092-4782-80c2-959ecee53134") : secret "maas-api-e2e-ait-40966d85-serving-cert" not found
Oct 31 04:31:41.206750 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.206728    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.208708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.208683    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.251545 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.251508    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:41.269149 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.269115    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkc2p\" (UniqueName: \"kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.398186 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.398101    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:41.537447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.537405    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:41.717336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.717247    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.719825 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.719793    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\") pod \"maas-api-e2e-ait-40966d85-6864fb578c-x2ksc\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:41.981910 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:41.981824    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:42.356576 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:31:42.356519    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ee89bf2_5092_4782_80c2_959ecee53134.slice/crio-dde3848f8823170ead79bdb519beb1bbcdfcd01fed7057ef705c0b3cf688e2ac WatchSource:0}: Error finding container dde3848f8823170ead79bdb519beb1bbcdfcd01fed7057ef705c0b3cf688e2ac: Status 404 returned error can't find the container with id dde3848f8823170ead79bdb519beb1bbcdfcd01fed7057ef705c0b3cf688e2ac
Oct 31 04:31:42.358124 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:42.358100    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"]
Oct 31 04:31:42.954896 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:42.954857    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" event={"ID":"0ee89bf2-5092-4782-80c2-959ecee53134","Type":"ContainerStarted","Data":"dde3848f8823170ead79bdb519beb1bbcdfcd01fed7057ef705c0b3cf688e2ac"}
Oct 31 04:31:43.961007 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:43.960968    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" event={"ID":"0ee89bf2-5092-4782-80c2-959ecee53134","Type":"ContainerStarted","Data":"ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5"}
Oct 31 04:31:43.961435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:43.961098    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:44.084112 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:44.084053    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" podStartSLOduration=2.227645672 podStartE2EDuration="3.0840362s" totalImagesPullingTime="856.390528ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:31:44.081534151 +0000 UTC m=+3753.915394488" watchObservedRunningTime="2026-10-31 04:31:44.0840362 +0000 UTC m=+3753.917896546"
Oct 31 04:31:46.455984 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:46.455931    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:49.972444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:49.972414    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:31:51.010906 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:51.010872    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:55.504486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:55.504452    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:31:59.042153 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:59.042120    5906 generic.go:356] "Generic (PLEG): container finished" podID="5b524a97-36a2-4669-a250-566dace2e8ab" containerID="2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26" exitCode=0
Oct 31 04:31:59.042623 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:59.042182    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" event={"ID":"5b524a97-36a2-4669-a250-566dace2e8ab","Type":"ContainerDied","Data":"2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"}
Oct 31 04:31:59.042623 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:59.042599    5906 scope.go:122] "RemoveContainer" containerID="2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"
Oct 31 04:31:59.043936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:59.043908    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:31:59.044010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:59.043977    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:31:59.044052 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:31:59.044016    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:32:00.072169 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:00.072130    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" event={"ID":"5b524a97-36a2-4669-a250-566dace2e8ab","Type":"ContainerStarted","Data":"a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4"}
Oct 31 04:32:00.304656 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:00.304618    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:00.405405 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:00.405319    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:32:00.405567 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:00.405552    5906 patch_prober.go:29] interesting pod/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 31 04:32:00.405628 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:00.405606    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 31 04:32:01.405101 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:01.405056    5906 patch_prober.go:29] interesting pod/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 31 04:32:01.405554 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:01.405121    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 31 04:32:02.405619 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:02.405576    5906 patch_prober.go:29] interesting pod/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 31 04:32:02.406098 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:02.405638    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 31 04:32:03.404749 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:03.404714    5906 patch_prober.go:29] interesting pod/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 31 04:32:03.404940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:03.404772    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 31 04:32:04.405383 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:04.405347    5906 patch_prober.go:29] interesting pod/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 31 04:32:04.405789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:04.405405    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 31 04:32:04.802907 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:04.802858    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:05.409176 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:05.409143    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:32:05.409657 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:05.409491    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:32:05.410241 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:05.410194    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:32:06.382425 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:32:06.382387    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:32:09.403770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:09.403735    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:12.127136 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:12.127103    5906 scope.go:122] "RemoveContainer" containerID="f0d577f40014d5ccd42917db354ce2ddf66491f6a8bfbd786193006177af67fb"
Oct 31 04:32:12.136110 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:12.136086    5906 scope.go:122] "RemoveContainer" containerID="ad13a2d85c2ebffca1125a057c0c4af036ef4dda9855a5c3baf14aafe5da1086"
Oct 31 04:32:13.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:13.903194    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:18.401884 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:18.401850    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:23.409770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:23.409720    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:28.001817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:28.001772    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:31.196263 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:31.196197    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:31.335140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:31.335101    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:32.500768 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:32.500721    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:35.600508 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:35.600465    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:36.354290 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.354253    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"]
Oct 31 04:32:36.354508 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.354482    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" podUID="0ee89bf2-5092-4782-80c2-959ecee53134" containerName="maas-api" containerID="cri-o://ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5" gracePeriod=30
Oct 31 04:32:36.393572 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:32:36.393518    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:32:36.459435 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.459389    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:36.640509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.640482    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:32:36.698535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.698500    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p\" (UniqueName: \"kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p\") pod \"0ee89bf2-5092-4782-80c2-959ecee53134\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") "
Oct 31 04:32:36.698745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.698573    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs\") pod \"0ee89bf2-5092-4782-80c2-959ecee53134\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") "
Oct 31 04:32:36.698745 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.698647    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca\") pod \"0ee89bf2-5092-4782-80c2-959ecee53134\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") "
Oct 31 04:32:36.698857 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.698760    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\") pod \"0ee89bf2-5092-4782-80c2-959ecee53134\" (UID: \"0ee89bf2-5092-4782-80c2-959ecee53134\") "
Oct 31 04:32:36.699078 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.699052    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca" pod "0ee89bf2-5092-4782-80c2-959ecee53134" (UID: "0ee89bf2-5092-4782-80c2-959ecee53134"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:32:36.699268 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.699245    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ee89bf2-5092-4782-80c2-959ecee53134-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:36.700765 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.700734    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p" pod "0ee89bf2-5092-4782-80c2-959ecee53134" (UID: "0ee89bf2-5092-4782-80c2-959ecee53134"). InnerVolumeSpecName "kube-api-access-xkc2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:32:36.700861 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.700828    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs" pod "0ee89bf2-5092-4782-80c2-959ecee53134" (UID: "0ee89bf2-5092-4782-80c2-959ecee53134"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:32:36.701057 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.701023    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls" pod "0ee89bf2-5092-4782-80c2-959ecee53134" (UID: "0ee89bf2-5092-4782-80c2-959ecee53134"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:32:36.800420 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.800385    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:36.800420 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.800419    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkc2p\" (UniqueName: \"kubernetes.io/projected/0ee89bf2-5092-4782-80c2-959ecee53134-kube-api-access-xkc2p\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:36.800636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:36.800434    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ee89bf2-5092-4782-80c2-959ecee53134-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:37.329688 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.329649    5906 generic.go:356] "Generic (PLEG): container finished" podID="0ee89bf2-5092-4782-80c2-959ecee53134" containerID="ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5" exitCode=0
Oct 31 04:32:37.329898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.329699    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"
Oct 31 04:32:37.329898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.329697    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" event={"ID":"0ee89bf2-5092-4782-80c2-959ecee53134","Type":"ContainerDied","Data":"ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5"}
Oct 31 04:32:37.329898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.329815    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc" event={"ID":"0ee89bf2-5092-4782-80c2-959ecee53134","Type":"ContainerDied","Data":"dde3848f8823170ead79bdb519beb1bbcdfcd01fed7057ef705c0b3cf688e2ac"}
Oct 31 04:32:37.329898 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.329845    5906 scope.go:122] "RemoveContainer" containerID="ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5"
Oct 31 04:32:37.339539 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.339516    5906 scope.go:122] "RemoveContainer" containerID="ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5"
Oct 31 04:32:37.339792 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:32:37.339771    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5\": container with ID starting with ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5 not found: ID does not exist" containerID="ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5"
Oct 31 04:32:37.339859 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.339805    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5"} err="failed to get container status \"ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5\": rpc error: code = NotFound desc = could not find container \"ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5\": container with ID starting with ccf07e70c0d218dbee1b945d797b4873a7dea004005274ef0e29439a789b3ba5 not found: ID does not exist"
Oct 31 04:32:37.499905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.499870    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"]
Oct 31 04:32:37.551714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.551677    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-40966d85-6864fb578c-x2ksc"]
Oct 31 04:32:37.871789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:37.871755    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:38.154776 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:38.154693    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:38.351197 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:38.351156    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ee89bf2-5092-4782-80c2-959ecee53134" path="/var/lib/kubelet/pods/0ee89bf2-5092-4782-80c2-959ecee53134/volumes"
Oct 31 04:32:40.563013 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:40.562966    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:40.566351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:40.566320    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"]
Oct 31 04:32:40.566626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:40.566597    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy" containerID="cri-o://a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4" gracePeriod=30
Oct 31 04:32:41.858631 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:41.858579    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:42.415496 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.415457    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"]
Oct 31 04:32:42.415946 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.415932    5906 cpu_manager.go:414] "containerMap: removing container" podUID="0ee89bf2-5092-4782-80c2-959ecee53134" containerName="maas-api"
Oct 31 04:32:42.416001 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.415948    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ee89bf2-5092-4782-80c2-959ecee53134" containerName="maas-api"
Oct 31 04:32:42.416045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.416027    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ee89bf2-5092-4782-80c2-959ecee53134" containerName="maas-api"
Oct 31 04:32:42.420568 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.420547    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.482370 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.482338    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-bb6bc8-openshift-default-dockercfg-qt7xc\""
Oct 31 04:32:42.514429 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.514393    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:42.535513 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.535476    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"]
Oct 31 04:32:42.567140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567106    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567162    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567296    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567351 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567326    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c46kh\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567460 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567350    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567460 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567389    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567538 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567471    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567538 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567524    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.567612 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.567573    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668600    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668641    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c46kh\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668664    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668698    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668720    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668767    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668804    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668870    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.668992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.668933    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.669344 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.669059    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.669344 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.669182    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.669344 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.669274    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.669516 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.669409    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.669624 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.669595    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.671069 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.671047    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.671564 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.671541    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.732365 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.732332    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.732535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.732476    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c46kh\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh\") pod \"e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") " pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.734343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.734321    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:42.868680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:42.868642    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:43.157178 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:32:43.157142    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21c775df_45b3_4770_9c8f_00a2e8cb69e2.slice/crio-30223bb2496e60c1e20d3447ff00f3ceb14e2a610821737c1f77ec94e7c35ebe WatchSource:0}: Error finding container 30223bb2496e60c1e20d3447ff00f3ceb14e2a610821737c1f77ec94e7c35ebe: Status 404 returned error can't find the container with id 30223bb2496e60c1e20d3447ff00f3ceb14e2a610821737c1f77ec94e7c35ebe
Oct 31 04:32:43.158161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.158130    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"]
Oct 31 04:32:43.159459 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.159426    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:32:43.159541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.159492    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:32:43.159541 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.159522    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:32:43.364140 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.364101    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" event={"ID":"21c775df-45b3-4770-9c8f-00a2e8cb69e2","Type":"ContainerStarted","Data":"01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"}
Oct 31 04:32:43.364336 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.364166    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" event={"ID":"21c775df-45b3-4770-9c8f-00a2e8cb69e2","Type":"ContainerStarted","Data":"30223bb2496e60c1e20d3447ff00f3ceb14e2a610821737c1f77ec94e7c35ebe"}
Oct 31 04:32:43.489952 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.489832    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podStartSLOduration=1.4898179489999999 podStartE2EDuration="1.489817949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:32:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:32:43.488312896 +0000 UTC m=+3813.322173200" watchObservedRunningTime="2026-10-31 04:32:43.489817949 +0000 UTC m=+3813.323678253"
Oct 31 04:32:43.735072 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:43.735027    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:44.740274 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:44.740241    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:45.374276 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.374247    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:45.375233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.375183    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:32:45.826032 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.825998    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:32:45.907321 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907285    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907484 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907339    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907484 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907376    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907542    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907612    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907658    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:32:45.907687 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907671    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907756    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907804    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907849    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs\") pod \"5b524a97-36a2-4669-a250-566dace2e8ab\" (UID: \"5b524a97-36a2-4669-a250-566dace2e8ab\") "
Oct 31 04:32:45.907942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907887    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:32:45.907942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.907908    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:32:45.908245 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908190    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:32:45.908317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908248    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:32:45.908380 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908363    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5b524a97-36a2-4669-a250-566dace2e8ab-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:45.908443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908384    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:45.908443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908398    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:45.908443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908412    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:45.908443 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.908426    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:45.909795 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.909763    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:32:45.909924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.909871    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:32:45.909924 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.909898    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "kube-api-access-f64d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:32:45.910041 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:45.909970    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo" pod "5b524a97-36a2-4669-a250-566dace2e8ab" (UID: "5b524a97-36a2-4669-a250-566dace2e8ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:32:46.010014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.009976    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5b524a97-36a2-4669-a250-566dace2e8ab-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:46.010014 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.010012    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:46.010296 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.010025    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5b524a97-36a2-4669-a250-566dace2e8ab-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:46.010296 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.010037    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f64d6\" (UniqueName: \"kubernetes.io/projected/5b524a97-36a2-4669-a250-566dace2e8ab-kube-api-access-f64d6\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:32:46.380899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.380860    5906 generic.go:356] "Generic (PLEG): container finished" podID="5b524a97-36a2-4669-a250-566dace2e8ab" containerID="a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4" exitCode=0
Oct 31 04:32:46.381119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.380937    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"
Oct 31 04:32:46.381119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.380943    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" event={"ID":"5b524a97-36a2-4669-a250-566dace2e8ab","Type":"ContainerDied","Data":"a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4"}
Oct 31 04:32:46.381119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.381001    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj" event={"ID":"5b524a97-36a2-4669-a250-566dace2e8ab","Type":"ContainerDied","Data":"4da2cca7953bf531aad3295e5e42b1bfcb1515791c6179f67bfd83503abf0de9"}
Oct 31 04:32:46.381119 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.381025    5906 scope.go:122] "RemoveContainer" containerID="a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4"
Oct 31 04:32:46.390590 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.390567    5906 scope.go:122] "RemoveContainer" containerID="2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"
Oct 31 04:32:46.399920 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.399896    5906 scope.go:122] "RemoveContainer" containerID="a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4"
Oct 31 04:32:46.400246 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:32:46.400194    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4\": container with ID starting with a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4 not found: ID does not exist" containerID="a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4"
Oct 31 04:32:46.400339 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.400262    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4"} err="failed to get container status \"a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4\": rpc error: code = NotFound desc = could not find container \"a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4\": container with ID starting with a23df00421b9b09de6e5efcdc63b8ff48033048480770c33cee4ba2fdf36dee4 not found: ID does not exist"
Oct 31 04:32:46.400339 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.400290    5906 scope.go:122] "RemoveContainer" containerID="2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"
Oct 31 04:32:46.400574 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:32:46.400554    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26\": container with ID starting with 2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26 not found: ID does not exist" containerID="2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"
Oct 31 04:32:46.400632 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.400580    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26"} err="failed to get container status \"2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26\": rpc error: code = NotFound desc = could not find container \"2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26\": container with ID starting with 2a7c33dff54e22dc4c75ac44bceba95931ccc515354b94cf686dd300d3442b26 not found: ID does not exist"
Oct 31 04:32:46.551288 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.551250    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"]
Oct 31 04:32:46.602968 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:46.602933    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-40966d85-openshift-default-7d89c65bcc-qrjrj"]
Oct 31 04:32:48.351427 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:48.351390    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" path="/var/lib/kubelet/pods/5b524a97-36a2-4669-a250-566dace2e8ab/volumes"
Oct 31 04:32:56.239087 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:56.239012    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:56.441725 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:56.441674    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:59.338355 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:59.338328    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:32:59.497453 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:32:59.497418    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:02.066444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.066409    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"]
Oct 31 04:33:02.066883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.066851    5906 cpu_manager.go:414] "containerMap: removing container" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy"
Oct 31 04:33:02.066883 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.066861    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy"
Oct 31 04:33:02.066980 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.066950    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy"
Oct 31 04:33:02.066980 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.066963    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy"
Oct 31 04:33:02.072030 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.072007    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.082043 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.082003    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.082198 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.082058    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh28b\" (UniqueName: \"kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.082309 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.082290    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.082467 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.082441    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.126915 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.126877    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"]
Oct 31 04:33:02.176325 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.176297    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-bb6bc8-serving-cert\""
Oct 31 04:33:02.183642 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.183611    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.183802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.183649    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh28b\" (UniqueName: \"kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.183802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.183729    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.183802 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.183792    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.183933 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:33:02.183900    5906 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-serving-cert: secret "maas-api-e2e-authcfg-http-bb6bc8-serving-cert" not found
Oct 31 04:33:02.183985 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:33:02.183960    5906 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls podName:1a7152fb-20b3-4d68-9f22-cf5a73dd79b1 nodeName:}" failed. No retries permitted until 2026-10-31 04:33:02.683941336 +0000 UTC m=+3832.517801634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls") pod "maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" (UID: "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1") : secret "maas-api-e2e-authcfg-http-bb6bc8-serving-cert" not found
Oct 31 04:33:02.184533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.184507    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.187754 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.187706    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.241899 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.241870    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:02.248996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.248966    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh28b\" (UniqueName: \"kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.393922 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.393845    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:02.531474 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.531437    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:02.688323 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.688232    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.690659 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.690637    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:02.985445 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:02.985351    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:03.357944 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:33:03.357899    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a7152fb_20b3_4d68_9f22_cf5a73dd79b1.slice/crio-8c9a44d569e6ce23b4c74586fd0fa63f80cf56460b5e37a412206fc871aa4e28 WatchSource:0}: Error finding container 8c9a44d569e6ce23b4c74586fd0fa63f80cf56460b5e37a412206fc871aa4e28: Status 404 returned error can't find the container with id 8c9a44d569e6ce23b4c74586fd0fa63f80cf56460b5e37a412206fc871aa4e28
Oct 31 04:33:03.360603 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:03.360553    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"]
Oct 31 04:33:03.506473 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:03.506434    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" event={"ID":"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1","Type":"ContainerStarted","Data":"8c9a44d569e6ce23b4c74586fd0fa63f80cf56460b5e37a412206fc871aa4e28"}
Oct 31 04:33:04.513826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:04.513784    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" event={"ID":"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1","Type":"ContainerStarted","Data":"443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639"}
Oct 31 04:33:04.514347 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:04.513848    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:04.637251 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:04.637115    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" podStartSLOduration=1.7273732370000001 podStartE2EDuration="2.637100763s" totalImagesPullingTime="909.727526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:33:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:33:04.634881384 +0000 UTC m=+3834.468741688" watchObservedRunningTime="2026-10-31 04:33:04.637100763 +0000 UTC m=+3834.470961068"
Oct 31 04:33:06.381791 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:33:06.381746    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:33:06.899543 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:06.899505    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:10.524978 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:10.524948    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:33:11.500485 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:11.500445    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:12.166232 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:12.166148    5906 scope.go:122] "RemoveContainer" containerID="a23853e2e5eaf249daa3ef4c6a5b45f13d68136416218f33449d29a51024ce85"
Oct 31 04:33:15.995447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:15.995409    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:18.615626 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:18.615588    5906 generic.go:356] "Generic (PLEG): container finished" podID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerID="01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2" exitCode=0
Oct 31 04:33:18.616174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:18.615662    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" event={"ID":"21c775df-45b3-4770-9c8f-00a2e8cb69e2","Type":"ContainerDied","Data":"01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"}
Oct 31 04:33:18.616174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:18.616065    5906 scope.go:122] "RemoveContainer" containerID="01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"
Oct 31 04:33:18.617388 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:18.617362    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:33:18.617486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:18.617427    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:33:18.617486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:18.617458    5906 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"}
Oct 31 04:33:19.622817 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:19.622781    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" event={"ID":"21c775df-45b3-4770-9c8f-00a2e8cb69e2","Type":"ContainerStarted","Data":"e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d"}
Oct 31 04:33:19.735172 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:19.735139    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:33:19.735391 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:19.735355    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 31 04:33:19.735490 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:19.735402    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 31 04:33:20.735766 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:20.735725    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 31 04:33:20.736196 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:20.735783    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 31 04:33:20.754877 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:20.754836    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:21.138246 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:21.138191    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:21.734969 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:21.734928    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 31 04:33:21.735167 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:21.734994    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 31 04:33:22.735382 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:22.735343    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 31 04:33:22.735900 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:22.735419    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 31 04:33:23.735512 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:23.735471    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 31 04:33:23.735927 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:23.735546    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 31 04:33:24.735178 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:24.735139    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 31 04:33:24.735407 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:24.735243    5906 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 31 04:33:25.197304 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:25.197274    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:25.740618 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:25.740586    5906 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:33:25.740941 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:25.740918    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:33:25.741698 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:25.741680    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:33:29.701536 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:29.701501    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:34.203884 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:34.203841    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:35.946636 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:35.946599    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:36.382457 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:33:36.382421    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:33:37.150181 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:37.150148    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:37.630975 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:37.630936    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:38.724499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:38.724464    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:43.198696 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:43.198655    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:47.703235 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:47.703173    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:52.216025 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:52.215938    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:33:56.700992 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:33:56.700954    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:01.198422 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:01.198384    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:05.704625 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:05.704576    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:06.381681 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:34:06.381647    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:34:09.426656 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:09.426624    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:10.200333 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:10.200287    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:14.692369 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:14.692332    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:19.202509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:19.202476    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:23.855487 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:23.855453    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:28.293161 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:28.293131    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:32.802864 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:32.802818    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:36.336550 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:36.336507    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:36.385776 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:34:36.385730    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:34:37.299953 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:37.299919    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:41.806651 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:41.806614    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:46.296774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:46.296742    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:50.809138 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:50.809097    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:34:55.493543 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:34:55.493506    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:00.116040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:00.115995    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:03.602913 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:03.602875    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:03.745019 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:03.744981    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:04.606553 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:04.606521    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:06.380944 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:35:06.380905    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:35:13.531139 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.531102    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"]
Oct 31 04:35:13.531779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.531334    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" podUID="1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" containerName="maas-api" containerID="cri-o://443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639" gracePeriod=30
Oct 31 04:35:13.613777 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.613740    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:13.808556 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.808519    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:35:13.912081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.912045    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b\" (UniqueName: \"kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b\") pod \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") "
Oct 31 04:35:13.912291 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.912145    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\") pod \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") "
Oct 31 04:35:13.912291 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.912242    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca\") pod \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") "
Oct 31 04:35:13.912291 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.912279    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs\") pod \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\" (UID: \"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1\") "
Oct 31 04:35:13.913223 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.913177    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca" pod "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" (UID: "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:35:13.914739 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.914704    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls" pod "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" (UID: "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:35:13.914863 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.914784    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs" pod "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" (UID: "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:35:13.914863 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:13.914818    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b" pod "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" (UID: "1a7152fb-20b3-4d68-9f22-cf5a73dd79b1"). InnerVolumeSpecName "kube-api-access-lh28b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:35:14.013444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.013402    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh28b\" (UniqueName: \"kubernetes.io/projected/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-kube-api-access-lh28b\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:14.013444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.013441    5906 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-maas-api-tls\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:14.013444 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.013452    5906 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-openshift-service-ca\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:14.013720 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.013461    5906 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1-metrics-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:14.220090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.219987    5906 generic.go:356] "Generic (PLEG): container finished" podID="1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" containerID="443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639" exitCode=0
Oct 31 04:35:14.220090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.220033    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"
Oct 31 04:35:14.220090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.220072    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" event={"ID":"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1","Type":"ContainerDied","Data":"443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639"}
Oct 31 04:35:14.220416 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.220140    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz" event={"ID":"1a7152fb-20b3-4d68-9f22-cf5a73dd79b1","Type":"ContainerDied","Data":"8c9a44d569e6ce23b4c74586fd0fa63f80cf56460b5e37a412206fc871aa4e28"}
Oct 31 04:35:14.220416 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.220161    5906 scope.go:122] "RemoveContainer" containerID="443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639"
Oct 31 04:35:14.234657 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.234633    5906 scope.go:122] "RemoveContainer" containerID="443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639"
Oct 31 04:35:14.234999 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:35:14.234980    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639\": container with ID starting with 443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639 not found: ID does not exist" containerID="443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639"
Oct 31 04:35:14.235081 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.235014    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639"} err="failed to get container status \"443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639\": rpc error: code = NotFound desc = could not find container \"443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639\": container with ID starting with 443df25f8941ae32d8a1fd0fddb2f58fe9ce0b173f1c2c12d6975a7870018639 not found: ID does not exist"
Oct 31 04:35:14.402428 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.402386    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"]
Oct 31 04:35:14.402651 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:14.402446    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-bb6bc8-66544f6d7-mwfxz"]
Oct 31 04:35:15.016485 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:15.016451    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:15.284266 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:15.284177    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:16.081058 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:16.081016    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:16.351386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:16.351308    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" path="/var/lib/kubelet/pods/1a7152fb-20b3-4d68-9f22-cf5a73dd79b1/volumes"
Oct 31 04:35:22.440269 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:22.440148    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:22.581472 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:22.581431    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:22.786598 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:22.786563    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"]
Oct 31 04:35:22.786849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:22.786812    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" containerID="cri-o://e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d" gracePeriod=30
Oct 31 04:35:22.792120 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:22.792095    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:25.743993 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:25.743949    5906 patch_prober.go:29] interesting pod/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:35:25.744404 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:25.744017    5906 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:35:28.047106 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.047073    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:35:28.079174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079088    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079174 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079135    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079188    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079242    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079331    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079395 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079368    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079434    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079445    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:35:28.079577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079487    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079541    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data\") pod \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\" (UID: \"21c775df-45b3-4770-9c8f-00a2e8cb69e2\") "
Oct 31 04:35:28.079811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079711    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:35:28.079811 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079739    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:35:28.079939 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.079895    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:35:28.080041 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.080024    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.080122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.080044    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.080122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.080058    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.080122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.080060    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:35:28.080122 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.080072    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.081666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.081636    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:35:28.081787 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.081698    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:35:28.081970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.081946    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "kube-api-access-c46kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:35:28.081970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.081946    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo" pod "21c775df-45b3-4770-9c8f-00a2e8cb69e2" (UID: "21c775df-45b3-4770-9c8f-00a2e8cb69e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:35:28.181476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.181438    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c46kh\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-kube-api-access-c46kh\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.181476 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.181476    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.181714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.181490    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.181714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.181499    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21c775df-45b3-4770-9c8f-00a2e8cb69e2-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.181714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.181507    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21c775df-45b3-4770-9c8f-00a2e8cb69e2-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:35:28.301033 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.300998    5906 generic.go:356] "Generic (PLEG): container finished" podID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerID="e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d" exitCode=0
Oct 31 04:35:28.301250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.301063    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"
Oct 31 04:35:28.301250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.301075    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" event={"ID":"21c775df-45b3-4770-9c8f-00a2e8cb69e2","Type":"ContainerDied","Data":"e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d"}
Oct 31 04:35:28.301250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.301137    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9" event={"ID":"21c775df-45b3-4770-9c8f-00a2e8cb69e2","Type":"ContainerDied","Data":"30223bb2496e60c1e20d3447ff00f3ceb14e2a610821737c1f77ec94e7c35ebe"}
Oct 31 04:35:28.301250 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.301167    5906 scope.go:122] "RemoveContainer" containerID="e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d"
Oct 31 04:35:28.311043 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.311019    5906 scope.go:122] "RemoveContainer" containerID="01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"
Oct 31 04:35:28.319652 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.319629    5906 scope.go:122] "RemoveContainer" containerID="e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d"
Oct 31 04:35:28.319938 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:35:28.319918    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d\": container with ID starting with e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d not found: ID does not exist" containerID="e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d"
Oct 31 04:35:28.319990 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.319949    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d"} err="failed to get container status \"e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d\": rpc error: code = NotFound desc = could not find container \"e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d\": container with ID starting with e4bdf805df83a7ac23d0f1dac434f94c60550224d541817b3b348bc54aaae84d not found: ID does not exist"
Oct 31 04:35:28.319990 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.319972    5906 scope.go:122] "RemoveContainer" containerID="01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"
Oct 31 04:35:28.320218 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:35:28.320187    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2\": container with ID starting with 01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2 not found: ID does not exist" containerID="01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"
Oct 31 04:35:28.320346 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.320327    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2"} err="failed to get container status \"01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2\": rpc error: code = NotFound desc = could not find container \"01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2\": container with ID starting with 01a0e1b571249f05560537a4409e700711e68949323539a52e8d4ea343e617c2 not found: ID does not exist"
Oct 31 04:35:28.472294 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.472190    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"]
Oct 31 04:35:28.475789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:28.475756    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-bb6bc8-openshift-default-85bb97c6d9-jhjd9"]
Oct 31 04:35:30.351508 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:30.351474    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" path="/var/lib/kubelet/pods/21c775df-45b3-4770-9c8f-00a2e8cb69e2/volumes"
Oct 31 04:35:36.382458 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:35:36.382411    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:35:36.641466 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:36.641383    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:35:49.049073 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:49.049042    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"]
Oct 31 04:35:49.100449 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:49.100414    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-8qn7j"]
Oct 31 04:35:50.351467 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:50.351427    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b001c13e-328d-4598-9a8c-43727b8b9837" path="/var/lib/kubelet/pods/b001c13e-328d-4598-9a8c-43727b8b9837/volumes"
Oct 31 04:35:53.345770 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:35:53.345733    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:36:04.989960 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:04.989923    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:36:05.809374 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:05.809336    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:36:06.383081 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:36:06.383048    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:36:17.049774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:17.049737    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"]
Oct 31 04:36:17.100942 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:17.100910    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3a0f9d5-htj7m"]
Oct 31 04:36:18.351039 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:18.350990    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab887103-bf92-47b2-8f92-bb112d545045" path="/var/lib/kubelet/pods/ab887103-bf92-47b2-8f92-bb112d545045/volumes"
Oct 31 04:36:36.568479 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:36:36.568434    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:36:37.023996 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:37.023865    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:36:56.911616 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:56.911527    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-4cxnd_e95ce020-c6a0-415f-af9c-52d50352f106/manager/0.log"
Oct 31 04:36:59.728037 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:36:59.728005    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c8b55458c-887dn_a3475cd3-fd85-4b01-8e83-b449bd11a898/maas-api/0.log"
Oct 31 04:37:01.047261 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:01.047221    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-5qnvm_948ce7a9-1e54-40aa-b7af-8d8a90567780/postgres/0.log"
Oct 31 04:37:03.506252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:03.506201    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2hbr7_410a60bd-b93a-458e-a814-3b3a01d4b074/kuadrant-console-plugin/0.log"
Oct 31 04:37:05.335026 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:05.334987    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-qns5d_e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc/discovery/0.log"
Oct 31 04:37:06.382103 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:37:06.382066    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:37:08.544341 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:08.544306    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:37:12.234097 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:12.234061    5906 scope.go:122] "RemoveContainer" containerID="4b49825b8afa67039937c149a4378e6edc71db38def237b87126d923a6c54386"
Oct 31 04:37:15.687382 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:15.687351    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5897747699-5zggr_c7b42a40-d70d-4b02-8bac-1056be986ecc/kube-auth-proxy/0.log"
Oct 31 04:37:16.013575 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:16.013541    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:37:16.144192 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:16.144161    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-4wtlq_6a1ab82b-5c52-4346-8fdf-fbf96210676f/istio-proxy/0.log"
Oct 31 04:37:16.149707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:16.149680    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:37:16.418077 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:16.417995    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-64ln7_a5f21edb-6647-45a7-a8db-800cf4748bea/istio-proxy/0.log"
Oct 31 04:37:18.005414 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:18.005383    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-rx2xb_67ef6a93-1fe0-44d4-9532-953a9b58dbb1/istio-proxy/0.log"
Oct 31 04:37:18.558820 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:18.558790    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-zcwhh_a4c1369a-ae0c-476f-b9ea-b31457b9e97b/istio-proxy/0.log"
Oct 31 04:37:36.382552 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:37:36.382514    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:37:37.359697 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:37.359655    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:37:40.810233 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:40.810167    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-mj8ld_017390dc-370f-4bd0-ae0e-0340ebf8b2cb/main/0.log"
Oct 31 04:37:41.036186 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:41.036137    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-ljbgs_f66cfd64-9d29-4813-86a2-4b0e766fae49/main/0.log"
Oct 31 04:37:41.566447 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:41.566415    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c_67694bd7-6e2d-42ac-b33b-befb107864a8/main/0.log"
Oct 31 04:37:42.046130 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:37:42.046099    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc_86ae1d89-4330-42ac-ba91-977fdb19878e/main/0.log"
Oct 31 04:38:06.382011 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:38:06.381970    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:38:12.257173 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:12.257131    5906 scope.go:122] "RemoveContainer" containerID="404ca826066241960fbbc340e7acf7aca0983697625d2fa547de065188c88d65"
Oct 31 04:38:13.887337 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:13.887304    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-4cxnd_e95ce020-c6a0-415f-af9c-52d50352f106/manager/0.log"
Oct 31 04:38:16.265413 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:16.265380    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c8b55458c-887dn_a3475cd3-fd85-4b01-8e83-b449bd11a898/maas-api/0.log"
Oct 31 04:38:17.648507 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:17.648470    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-5qnvm_948ce7a9-1e54-40aa-b7af-8d8a90567780/postgres/0.log"
Oct 31 04:38:36.384993 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:38:36.384944    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:38:37.708706 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:37.708667    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:38:39.847678 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:39.847647    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2hbr7_410a60bd-b93a-458e-a814-3b3a01d4b074/kuadrant-console-plugin/0.log"
Oct 31 04:38:41.708970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:41.708932    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-qns5d_e72c8692-ad29-4343-a3dd-5f5ca3cc4fdc/discovery/0.log"
Oct 31 04:38:42.083690 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:42.083660    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5897747699-5zggr_c7b42a40-d70d-4b02-8bac-1056be986ecc/kube-auth-proxy/0.log"
Oct 31 04:38:42.610535 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:42.610506    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-4wtlq_6a1ab82b-5c52-4346-8fdf-fbf96210676f/istio-proxy/0.log"
Oct 31 04:38:42.849747 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:42.849715    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-64ln7_a5f21edb-6647-45a7-a8db-800cf4748bea/istio-proxy/0.log"
Oct 31 04:38:44.541600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:44.541564    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-rx2xb_67ef6a93-1fe0-44d4-9532-953a9b58dbb1/istio-proxy/0.log"
Oct 31 04:38:55.145190 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:55.145153    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-zcwhh_a4c1369a-ae0c-476f-b9ea-b31457b9e97b/istio-proxy/0.log"
Oct 31 04:38:57.081450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:57.081419    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-mj8ld_017390dc-370f-4bd0-ae0e-0340ebf8b2cb/main/0.log"
Oct 31 04:38:57.357146 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:57.357056    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-ljbgs_f66cfd64-9d29-4813-86a2-4b0e766fae49/main/0.log"
Oct 31 04:38:57.926753 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:57.926725    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8zjf9c_67694bd7-6e2d-42ac-b33b-befb107864a8/main/0.log"
Oct 31 04:38:58.433148 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:38:58.433118    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-tsmkc_86ae1d89-4330-42ac-ba91-977fdb19878e/main/0.log"
Oct 31 04:39:06.381650 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:39:06.381612    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:39:17.814872 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:39:17.814841    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:39:36.382264 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:39:36.382188    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:39:38.166975 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:39:38.166942    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:40:06.381037 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:40:06.380953    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:40:36.382328 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:40:36.382272    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:40:38.418951 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:40:38.418916    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:41:06.381705 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:41:06.381661    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:41:36.381816 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:41:36.381734    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:41:38.766905 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:38.766869    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:41:58.842826 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.842790    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"]
Oct 31 04:41:58.843415 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.843003    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy" containerID="cri-o://ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f" gracePeriod=30
Oct 31 04:41:58.855920 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.855888    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"]
Oct 31 04:41:58.856132 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.856097    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy" containerID="cri-o://15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a" gracePeriod=30
Oct 31 04:41:58.857271 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.857249    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"]
Oct 31 04:41:58.857450 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.857417    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy" containerID="cri-o://dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142" gracePeriod=30
Oct 31 04:41:58.894940 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.894904    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"]
Oct 31 04:41:58.895170 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:41:58.895146    5906 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy" containerID="cri-o://61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a" gracePeriod=30
Oct 31 04:42:04.150919 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.150888    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:42:04.186609 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.186582    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:42:04.214465 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.214439    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:42:04.227442 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.227409    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:42:04.296921 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.296881    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.296944    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.296986    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297027    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297058    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297087    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297118    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297151    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297239    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297270 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297274    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297287    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297306    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297310    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297335    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297364    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297393    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297423    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297455    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297465    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297589    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297627    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297678    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297704    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297752    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297792    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.297849 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297835    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297874    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297919    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297958    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297992    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298049    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298092    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token\") pod \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\" (UID: \"67ef6a93-1fe0-44d4-9532-953a9b58dbb1\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298125    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298151    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298180    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298282    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298336    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket\") pod \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\" (UID: \"6a1ab82b-5c52-4346-8fdf-fbf96210676f\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298373    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs\") pod \"a5f21edb-6647-45a7-a8db-800cf4748bea\" (UID: \"a5f21edb-6647-45a7-a8db-800cf4748bea\") "
Oct 31 04:42:04.298774 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298413    5906 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket\") pod \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\" (UID: \"a4c1369a-ae0c-476f-b9ea-b31457b9e97b\") "
Oct 31 04:42:04.299533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298946    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.299533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298970    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.299533 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299000    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.299707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297575    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297633    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297671    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299707 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297741    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297771    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.297875    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298160    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.298880    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299186    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299386    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299434    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299451    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299582    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299650    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.299945 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299816    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:04.300599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.299956    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.300599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.300482    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.300599 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.300586    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.301468 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.301438    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "kube-api-access-hxdl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.302524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.302495    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "kube-api-access-c8zz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.302524 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.302509    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.303499 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303475    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.303625 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303561    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:04.303625 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303566    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.303779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303703    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "kube-api-access-r4k66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.303779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303707    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:04.303779 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303615    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.304505 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303764    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.304666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.303762    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo" pod "a4c1369a-ae0c-476f-b9ea-b31457b9e97b" (UID: "a4c1369a-ae0c-476f-b9ea-b31457b9e97b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:04.304666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.304158    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.304666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.304180    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm" pod "67ef6a93-1fe0-44d4-9532-953a9b58dbb1" (UID: "67ef6a93-1fe0-44d4-9532-953a9b58dbb1"). InnerVolumeSpecName "kube-api-access-bgbfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.305669 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.305643    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy" pod "a5f21edb-6647-45a7-a8db-800cf4748bea" (UID: "a5f21edb-6647-45a7-a8db-800cf4748bea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.306611 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.306586    5906 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo" pod "6a1ab82b-5c52-4346-8fdf-fbf96210676f" (UID: "6a1ab82b-5c52-4346-8fdf-fbf96210676f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:04.376674 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.376638    5906 generic.go:356] "Generic (PLEG): container finished" podID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerID="15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a" exitCode=0
Oct 31 04:42:04.376853 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.376698    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"
Oct 31 04:42:04.376853 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.376825    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" event={"ID":"a5f21edb-6647-45a7-a8db-800cf4748bea","Type":"ContainerDied","Data":"15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a"}
Oct 31 04:42:04.376974 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.376926    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7" event={"ID":"a5f21edb-6647-45a7-a8db-800cf4748bea","Type":"ContainerDied","Data":"dae4557def4f94462fbc80f87664c0857c4190f6536d99ea57ca8bd6f332c946"}
Oct 31 04:42:04.377020 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.376973    5906 scope.go:122] "RemoveContainer" containerID="15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a"
Oct 31 04:42:04.378538 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.378511    5906 generic.go:356] "Generic (PLEG): container finished" podID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerID="dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142" exitCode=0
Oct 31 04:42:04.378708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.378562    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"
Oct 31 04:42:04.378708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.378636    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" event={"ID":"67ef6a93-1fe0-44d4-9532-953a9b58dbb1","Type":"ContainerDied","Data":"dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142"}
Oct 31 04:42:04.378708 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.378689    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb" event={"ID":"67ef6a93-1fe0-44d4-9532-953a9b58dbb1","Type":"ContainerDied","Data":"44927d55c580f97e7072f2aa90d05e37f4261fbbeb3a5fa3bbddf6575d0dcfd5"}
Oct 31 04:42:04.380105 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.380079    5906 generic.go:356] "Generic (PLEG): container finished" podID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerID="ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f" exitCode=0
Oct 31 04:42:04.380375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.380176    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" event={"ID":"6a1ab82b-5c52-4346-8fdf-fbf96210676f","Type":"ContainerDied","Data":"ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f"}
Oct 31 04:42:04.380375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.380241    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq" event={"ID":"6a1ab82b-5c52-4346-8fdf-fbf96210676f","Type":"ContainerDied","Data":"be88bd87a13cf64f878330f611dfd9416330122b526c5fc2b9928ff704b0c5df"}
Oct 31 04:42:04.380375 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.380296    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"
Oct 31 04:42:04.382390 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.382364    5906 generic.go:356] "Generic (PLEG): container finished" podID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerID="61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a" exitCode=0
Oct 31 04:42:04.382509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.382399    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" event={"ID":"a4c1369a-ae0c-476f-b9ea-b31457b9e97b","Type":"ContainerDied","Data":"61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a"}
Oct 31 04:42:04.382509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.382418    5906 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"
Oct 31 04:42:04.382509 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.382456    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh" event={"ID":"a4c1369a-ae0c-476f-b9ea-b31457b9e97b","Type":"ContainerDied","Data":"9137e659b20377e9815eb26b862c513dbcb80efa3f5a2387c924a09d047b93be"}
Oct 31 04:42:04.388939 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.388915    5906 scope.go:122] "RemoveContainer" containerID="15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a"
Oct 31 04:42:04.389307 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:42:04.389284    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a\": container with ID starting with 15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a not found: ID does not exist" containerID="15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a"
Oct 31 04:42:04.389396 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.389315    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a"} err="failed to get container status \"15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a\": rpc error: code = NotFound desc = could not find container \"15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a\": container with ID starting with 15b9f5ce66d94db97c4dc5ddf9d511d3b1e7d15a39ae5700891f2896e963f99a not found: ID does not exist"
Oct 31 04:42:04.389396 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.389335    5906 scope.go:122] "RemoveContainer" containerID="dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142"
Oct 31 04:42:04.399360 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399329    5906 scope.go:122] "RemoveContainer" containerID="dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142"
Oct 31 04:42:04.399679 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:42:04.399653    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142\": container with ID starting with dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142 not found: ID does not exist" containerID="dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142"
Oct 31 04:42:04.399739 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399688    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142"} err="failed to get container status \"dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142\": rpc error: code = NotFound desc = could not find container \"dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142\": container with ID starting with dbeda5d6a85dda30c0e8684eecca488616179a377143d0203f4967c717f84142 not found: ID does not exist"
Oct 31 04:42:04.399739 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399707    5906 scope.go:122] "RemoveContainer" containerID="ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f"
Oct 31 04:42:04.399886 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399866    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.399936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399888    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.399936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399906    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.399936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399920    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.399936 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399934    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399944    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399953    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399961    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399975    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.399988    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400001    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxdl2\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-kube-api-access-hxdl2\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400014    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400028    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400041    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400053    5906 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-data\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400065    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400090 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400078    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8zz2\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-kube-api-access-c8zz2\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400094    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a1ab82b-5c52-4346-8fdf-fbf96210676f-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400110    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4k66\" (UniqueName: \"kubernetes.io/projected/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-kube-api-access-r4k66\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400124    5906 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgbfm\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-kube-api-access-bgbfm\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400137    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400150    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400163    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400177    5906 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-podinfo\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400190    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67ef6a93-1fe0-44d4-9532-953a9b58dbb1-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400219    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400233    5906 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5f21edb-6647-45a7-a8db-800cf4748bea-istiod-ca-cert\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400243    5906 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-istio-envoy\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400254    5906 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-workload-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400265    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a1ab82b-5c52-4346-8fdf-fbf96210676f-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400273    5906 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5f21edb-6647-45a7-a8db-800cf4748bea-workload-certs\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400281    5906 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4c1369a-ae0c-476f-b9ea-b31457b9e97b-credential-socket\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.400577 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.400295    5906 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5f21edb-6647-45a7-a8db-800cf4748bea-istio-token\") on node \"ip-10-0-129-108.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.408416 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.408393    5906 scope.go:122] "RemoveContainer" containerID="ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f"
Oct 31 04:42:04.408677 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:42:04.408660    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f\": container with ID starting with ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f not found: ID does not exist" containerID="ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f"
Oct 31 04:42:04.408732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.408688    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f"} err="failed to get container status \"ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f\": rpc error: code = NotFound desc = could not find container \"ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f\": container with ID starting with ce9d2051154bca9b4ff34a7f4987b3b8bafe7f2bcebe0f5b3a08ba1a6e7d442f not found: ID does not exist"
Oct 31 04:42:04.408732 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.408707    5906 scope.go:122] "RemoveContainer" containerID="61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a"
Oct 31 04:42:04.417073 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.417051    5906 scope.go:122] "RemoveContainer" containerID="61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a"
Oct 31 04:42:04.417491 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:42:04.417462    5906 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a\": container with ID starting with 61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a not found: ID does not exist" containerID="61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a"
Oct 31 04:42:04.417576 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.417502    5906 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a"} err="failed to get container status \"61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a\": rpc error: code = NotFound desc = could not find container \"61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a\": container with ID starting with 61e13f0cef2bc3a56f52ca0d2917b58bfb82e0015641a88d0f291032229aa50a not found: ID does not exist"
Oct 31 04:42:04.814714 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.814677    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"]
Oct 31 04:42:04.818666 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.818638    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-4wtlq"]
Oct 31 04:42:04.931396 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.931357    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"]
Oct 31 04:42:04.935040 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:04.935007    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-64ln7"]
Oct 31 04:42:05.048595 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:05.048558    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"]
Oct 31 04:42:05.052371 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:05.052341    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-rx2xb"]
Oct 31 04:42:05.165648 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:05.165551    5906 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"]
Oct 31 04:42:05.169480 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:05.169454    5906 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-zcwhh"]
Oct 31 04:42:06.352852 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:06.352814    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" path="/var/lib/kubelet/pods/67ef6a93-1fe0-44d4-9532-953a9b58dbb1/volumes"
Oct 31 04:42:06.353359 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:06.353343    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" path="/var/lib/kubelet/pods/6a1ab82b-5c52-4346-8fdf-fbf96210676f/volumes"
Oct 31 04:42:06.353784 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:06.353770    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" path="/var/lib/kubelet/pods/a4c1369a-ae0c-476f-b9ea-b31457b9e97b/volumes"
Oct 31 04:42:06.354571 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:06.354554    5906 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" path="/var/lib/kubelet/pods/a5f21edb-6647-45a7-a8db-800cf4748bea/volumes"
Oct 31 04:42:06.381891 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:42:06.381852    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:42:25.061369 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:25.061330    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:42:32.779789 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:32.779754    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:42:32.923136 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:32.923097    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:42:36.382005 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:42:36.381965    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:42:39.099503 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:39.099462    5906 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log" limit=1024
Oct 31 04:42:45.678783 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:45.678733    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xgj98_24753e34-7be9-4a7e-8629-622b22fde606/global-pull-secret-syncer/0.log"
Oct 31 04:42:47.551558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:47.551520    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p8r6z_c04cb90b-5446-4325-a368-a54295d49f33/konnectivity-agent/0.log"
Oct 31 04:42:48.888493 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:48.888410    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-108.us-west-2.compute.internal_79d038c1345182a8a45cdaeaa396f9d8/haproxy/0.log"
Oct 31 04:42:54.402909 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.402876    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-2hbr7_410a60bd-b93a-458e-a814-3b3a01d4b074/kuadrant-console-plugin/0.log"
Oct 31 04:42:54.650317 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650270    5906 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"]
Oct 31 04:42:54.650717 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650701    5906 cpu_manager.go:414] "containerMap: removing container" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650720    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650730    5906 cpu_manager.go:414] "containerMap: removing container" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650736    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650758    5906 cpu_manager.go:414] "containerMap: removing container" podUID="1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" containerName="maas-api"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650763    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" containerName="maas-api"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650770    5906 cpu_manager.go:414] "containerMap: removing container" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy"
Oct 31 04:42:54.650773 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650775    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650784    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650789    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650805    5906 cpu_manager.go:414] "containerMap: removing container" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650811    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650818    5906 cpu_manager.go:414] "containerMap: removing container" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650823    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650831    5906 cpu_manager.go:414] "containerMap: removing container" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650835    5906 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b524a97-36a2-4669-a250-566dace2e8ab" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650900    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650909    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="21c775df-45b3-4770-9c8f-00a2e8cb69e2" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650917    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a7152fb-20b3-4d68-9f22-cf5a73dd79b1" containerName="maas-api"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650923    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="67ef6a93-1fe0-44d4-9532-953a9b58dbb1" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650928    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f21edb-6647-45a7-a8db-800cf4748bea" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650934    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a1ab82b-5c52-4346-8fdf-fbf96210676f" containerName="istio-proxy"
Oct 31 04:42:54.651010 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.650939    5906 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4c1369a-ae0c-476f-b9ea-b31457b9e97b" containerName="istio-proxy"
Oct 31 04:42:54.654058 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.654007    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.708970 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.708929    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"]
Oct 31 04:42:54.758816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.758780    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xddgj\"/\"openshift-service-ca.crt\""
Oct 31 04:42:54.758816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.758802    5906 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xddgj\"/\"kube-root-ca.crt\""
Oct 31 04:42:54.758816 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.758802    5906 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xddgj\"/\"default-dockercfg-x59w7\""
Oct 31 04:42:54.794343 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.794303    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-sys\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.794558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.794364    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-podres\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.794558 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.794489    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kdn4\" (UniqueName: \"kubernetes.io/projected/b3d8c390-c5f3-461f-ab80-f3ca01cba454-kube-api-access-7kdn4\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.794680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.794611    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-proc\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.794680 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.794641    5906 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-lib-modules\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.902742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.902697    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-proc\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.902742 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.902740    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-proc\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.902762    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-lib-modules\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.903032    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-sys\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.902863    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-lib-modules\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.903098    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-sys\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.903145    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-podres\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903252 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.903184    5906 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kdn4\" (UniqueName: \"kubernetes.io/projected/b3d8c390-c5f3-461f-ab80-f3ca01cba454-kube-api-access-7kdn4\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.903603 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.903312    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b3d8c390-c5f3-461f-ab80-f3ca01cba454-podres\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.966049 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.965978    5906 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kdn4\" (UniqueName: \"kubernetes.io/projected/b3d8c390-c5f3-461f-ab80-f3ca01cba454-kube-api-access-7kdn4\") pod \"perf-node-gather-daemonset-n8rsn\" (UID: \"b3d8c390-c5f3-461f-ab80-f3ca01cba454\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:54.966840 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:54.966817    5906 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:55.343783 ip-10-0-129-108 kubenswrapper[5906]: W1031 04:42:55.343730    5906 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb3d8c390_c5f3_461f_ab80_f3ca01cba454.slice/crio-5154ae43b626914d5f47aba6d1d6fa9f1bfdc694651cfbed014a5ae488d14712 WatchSource:0}: Error finding container 5154ae43b626914d5f47aba6d1d6fa9f1bfdc694651cfbed014a5ae488d14712: Status 404 returned error can't find the container with id 5154ae43b626914d5f47aba6d1d6fa9f1bfdc694651cfbed014a5ae488d14712
Oct 31 04:42:55.344523 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:55.344487    5906 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"]
Oct 31 04:42:55.624723 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:55.624624    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn" event={"ID":"b3d8c390-c5f3-461f-ab80-f3ca01cba454","Type":"ContainerStarted","Data":"4f61d63ae7d184d912dee3a94b068e32bbb3286c3229891782cbbe930e02de36"}
Oct 31 04:42:55.624723 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:55.624682    5906 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn" event={"ID":"b3d8c390-c5f3-461f-ab80-f3ca01cba454","Type":"ContainerStarted","Data":"5154ae43b626914d5f47aba6d1d6fa9f1bfdc694651cfbed014a5ae488d14712"}
Oct 31 04:42:55.625150 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:55.624720    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:42:55.744098 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:55.744044    5906 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn" podStartSLOduration=1.744029608 podStartE2EDuration="1.744029608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 04:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:42:55.743876183 +0000 UTC m=+4425.577736487" watchObservedRunningTime="2026-10-31 04:42:55.744029608 +0000 UTC m=+4425.577889912"
Oct 31 04:42:58.090029 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:58.089997    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/alertmanager/0.log"
Oct 31 04:42:58.483348 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:58.483267    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/config-reloader/0.log"
Oct 31 04:42:58.874007 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:58.873979    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/kube-rbac-proxy-web/0.log"
Oct 31 04:42:59.263932 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:59.263906    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/kube-rbac-proxy/0.log"
Oct 31 04:42:59.654015 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:42:59.653921    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/kube-rbac-proxy-metric/0.log"
Oct 31 04:43:00.043452 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:00.043422    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/prom-label-proxy/0.log"
Oct 31 04:43:00.432045 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:00.431970    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_869b8227-2fd2-45e9-a10a-9bf27ba98d3f/init-config-reloader/0.log"
Oct 31 04:43:01.263102 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:01.263075    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-jsvx7_d0ca45c9-f001-466d-82b0-852e1e486661/kube-state-metrics/0.log"
Oct 31 04:43:01.640357 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:01.640287    5906 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-n8rsn"
Oct 31 04:43:01.647386 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:01.647355    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-jsvx7_d0ca45c9-f001-466d-82b0-852e1e486661/kube-rbac-proxy-main/0.log"
Oct 31 04:43:02.037217 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:02.037179    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-jsvx7_d0ca45c9-f001-466d-82b0-852e1e486661/kube-rbac-proxy-self/0.log"
Oct 31 04:43:02.431015 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:02.430934    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-79fcb4cc8-rjl9f_4dc6feeb-2db4-4605-b2d8-6220c8d27c9c/metrics-server/0.log"
Oct 31 04:43:04.397060 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:04.397031    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q5pft_35631c59-2c9c-4672-af14-6f7f6734fc24/node-exporter/0.log"
Oct 31 04:43:04.786096 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:04.786062    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q5pft_35631c59-2c9c-4672-af14-6f7f6734fc24/kube-rbac-proxy/0.log"
Oct 31 04:43:05.180926 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:05.180849    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-q5pft_35631c59-2c9c-4672-af14-6f7f6734fc24/init-textfile/0.log"
Oct 31 04:43:06.381318 ip-10-0-129-108 kubenswrapper[5906]: E1031 04:43:06.381279    5906 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gzcxv" containerName="extractor"
Oct 31 04:43:06.758214 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:06.758156    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-t6q8h_0acad573-0b62-497a-9068-546a37bbc5ea/kube-rbac-proxy-main/0.log"
Oct 31 04:43:07.150084 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:07.150010    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-t6q8h_0acad573-0b62-497a-9068-546a37bbc5ea/kube-rbac-proxy-self/0.log"
Oct 31 04:43:07.552882 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:07.552853    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-t6q8h_0acad573-0b62-497a-9068-546a37bbc5ea/openshift-state-metrics/0.log"
Oct 31 04:43:07.951838 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:07.951756    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/prometheus/0.log"
Oct 31 04:43:08.339700 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:08.339674    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/config-reloader/0.log"
Oct 31 04:43:08.729342 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:08.729262    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/thanos-sidecar/0.log"
Oct 31 04:43:09.119486 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:09.119459    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/kube-rbac-proxy-web/0.log"
Oct 31 04:43:09.507373 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:09.507342    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/kube-rbac-proxy/0.log"
Oct 31 04:43:09.901719 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:09.901641    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/kube-rbac-proxy-thanos/0.log"
Oct 31 04:43:10.290398 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:10.290366    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_072a8b97-c371-403e-981b-1c13e6d0080d/init-config-reloader/0.log"
Oct 31 04:43:10.687925 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:10.687840    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-shcbg_5cd7aa60-8222-467f-88e2-88cb44ff12a5/prometheus-operator/0.log"
Oct 31 04:43:11.081697 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:11.081665    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-shcbg_5cd7aa60-8222-467f-88e2-88cb44ff12a5/kube-rbac-proxy/0.log"
Oct 31 04:43:13.053686 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:13.053656    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8444c575f6-l95rj_6862e1ee-e114-483c-b37a-1ce7ddeb52aa/thanos-query/0.log"
Oct 31 04:43:13.445986 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:13.445895    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8444c575f6-l95rj_6862e1ee-e114-483c-b37a-1ce7ddeb52aa/kube-rbac-proxy-web/0.log"
Oct 31 04:43:13.835128 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:13.835092    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8444c575f6-l95rj_6862e1ee-e114-483c-b37a-1ce7ddeb52aa/kube-rbac-proxy/0.log"
Oct 31 04:43:14.224392 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:14.224308    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8444c575f6-l95rj_6862e1ee-e114-483c-b37a-1ce7ddeb52aa/prom-label-proxy/0.log"
Oct 31 04:43:14.614133 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:14.614101    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8444c575f6-l95rj_6862e1ee-e114-483c-b37a-1ce7ddeb52aa/kube-rbac-proxy-rules/0.log"
Oct 31 04:43:15.003600 ip-10-0-129-108 kubenswrapper[5906]: I1031 04:43:15.003572    5906 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8444c575f6-l95rj_6862e1ee-e114-483c-b37a-1ce7ddeb52aa/kube-rbac-proxy-metrics/0.log"